From owner-freebsd-net@FreeBSD.ORG Sun Sep 12 03:30:10 2010 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD7821065674 for ; Sun, 12 Sep 2010 03:30:10 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id AE6738FC19 for ; Sun, 12 Sep 2010 03:30:10 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o8C3UALX059321 for ; Sun, 12 Sep 2010 03:30:10 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o8C3UA03059318; Sun, 12 Sep 2010 03:30:10 GMT (envelope-from gnats) Date: Sun, 12 Sep 2010 03:30:10 GMT Message-Id: <201009120330.o8C3UA03059318@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: oliver.solaris@gmail.com Cc: Subject: Re: kern/143874: [wpi] Wireless 3945ABG error. wpi0 could not allocate memory resource X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: oliver.solaris@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Sep 2010 03:30:10 -0000 The following reply was made to PR kern/143874; it has been noted by GNATS. From: oliver.solaris@gmail.com> To: bug-followup@FreeBSD.org, kavolorn@gmail.com Cc: Subject: Re: kern/143874: [wpi] Wireless 3945ABG error. wpi0 could not allocate memory resource Date: Sat, 11 Sep 2010 21:55:53 -0500 I have the same issue with a Toshiba Satellite P105: Sep 11 21:46:13 kernel: wpi0: irq 17 at device 0.0 on pci3 Sep 11 21:46:13 kernel: wpi0: Driver Revision 20071127 Sep 11 21:46:13 kernel: wpi0: 0x1000 bytes of rid 0x10 res 3 failed (0, 0xfffff fff). Sep 11 21:46:13 kernel: wpi0: could not allocate memory resource Sep 11 21:46:13 kernel: device_attach: wpi0 attach returned 6 Please let me know if I can help out to track this issue down. From owner-freebsd-net@FreeBSD.ORG Sun Sep 12 04:57:00 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E15AE106566C for ; Sun, 12 Sep 2010 04:56:59 +0000 (UTC) (envelope-from sadishkr@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id AAE918FC0A for ; Sun, 12 Sep 2010 04:56:59 +0000 (UTC) Received: by iwn34 with SMTP id 34so4396307iwn.13 for ; Sat, 11 Sep 2010 21:56:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=MKZPow+PhFGfwfA66qDwql5Ul/2Bx6V7wv7Vs55hrBs=; b=K9NYNX9ohcF8AJEQK7wCzA1VDTBGxXQyZLPMzMtiYDtnUlVSQ2Esui5HALJmOPKL4k 63L5qgTbR6M8FJSj+op9AxCMFSKyF776eyX33lxm8vnQBDNTWl/+2uzo4uFUwBALurRA j++sRlagKL6+fa2Avb078pi3/8iWG/6oLuY+w= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=TBUl6y9ntCn3zk5iGe4DXuTo2oYVjWlxNcuOEz9KTpQHf0V3nGBe17+96AsY/k1kh5 MApi7XuPa0IclUmlAGEvrRYm183KNb6p9EPodFO4FViBwgY5pt/veqet3tNzXy9YZ5HH nNILT/e/73jLGquVR05Zv+MLb4VnkbcwXUN30= MIME-Version: 1.0 Received: by 10.231.11.197 with SMTP id u5mr3604194ibu.41.1284267418637; Sat, 11 Sep 2010 21:56:58 -0700 (PDT) Received: by 10.231.193.222 with HTTP; Sat, 11 Sep 2010 21:56:58 -0700 (PDT) Date: Sun, 12 Sep 2010 14:56:58 +1000 Message-ID: From: Sadish Kulasekere To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Creating socket(9) functions in FreeBSD 8 kernel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Sep 2010 04:57:00 -0000 Hi, I've been trying to write a program to send a message to a specific host with a specific MAC address. I used the socket(9) kernel interface socket functions, but the kernel keeps crashing when I try to load it. I have posted the code below. Can someone please tell me what is wrong with this code? Cheers, Sadish. #include //#include //#include #include //#include #include #include #include #include #include //#include #include #include #include #include #include #include #include #include #include #include #include #include #include //#include #include #include #include #include #include #include #include #include static int wake(void); //typedef u8; static int wake(void) { unsigned int i,j; unsigned char k[6] = {0x1F,0x00,0xDD,0x1D,0x3B,0x2E}; /*need to fix the correct MAC*/ unsigned char wolBuff[136]; unsigned char* ptr; //int optval = 1; int packet; //int sendint; struct socket *sok; struct sockaddr_in sap; //struct sockaddr *sap; //struct ucred *crd; //struct thread *trd; struct uio auio; struct iovec aiov; struct sockopt sokoptions; aiov.iov_base = (caddr_t) &wolBuff;//&wolBuff; aiov.iov_len = sizeof(wolBuff);//strlen(wolBuff); auio.uio_iov = &aiov; auio.uio_iovcnt = 1; auio.uio_offset = 0; auio.uio_resid = sizeof(wolBuff);//strlen(wolBuff); auio.uio_segflg = UIO_SYSSPACE; auio.uio_rw = UIO_WRITE; auio.uio_td = 0; //sokoptions.sopt_dir = SOPT_SET; //sokoptions.sopt_level = SOL_SOCKET; //sokoptions.sopt_name = packet; //sokoptions.sopt_val = 0; //sokoptions.sopt_valsize = 0; //sokoptions.sopt_td = NULL; ptr = wolBuff; for(i=0; i<6; i++) { *ptr++ = 0xFF; } for(j=0; j<16; j++) { for(i=0; i<6; i++) { *ptr++ = k[i]; } } //uprintf("print buffer %u",*ptr); //packet = socket (AF_INET, SOCK_DGRAM, IPPROTO_UDP); packet = socreate(AF_LINK, &sok, SOCK_DGRAM,0, 0, 0); if(sok==NULL) { uprintf("sok null\n"); goto ret; } sokoptions.sopt_dir = SOPT_SET; sokoptions.sopt_level = SOL_SOCKET; sokoptions.sopt_name = packet; sokoptions.sopt_val = 0; sokoptions.sopt_valsize = 0; sokoptions.sopt_td = NULL; //sosetopt((struct socket *)packet, &sokoptions); /* Set up broadcast address */ sap.sin_family = AF_INET; sap.sin_addr.s_addr = INADDR_BROADCAST;//htonl(0xffffffff) sap.sin_port = htons(6000); uprintf("aaaaaaaaaaaaa \n"); //just for testing packet = sosend(sok, (struct sockaddr *)&sap, &auio, 0, 0, 0, 0); if(packet <0) { uprintf("bbbbbbbbbbbbb \n"); goto ret; } //soclose(sok); ret: return 0; } static int deinit(void) { uprintf("module unloaded \n"); return 0; } /* The function to load/unload kernel module*/ static int event_handler(struct module *module, int event, void *arg) { int error =0; /* Error, 0 for normal return status*/ switch (event) { case MOD_LOAD: error = wake(); break; case MOD_UNLOAD: error = deinit(); break; default: error = EOPNOTSUPP; /*Error operation not supported*/ break; } return(error); } /*argument for DECLARE_MODULE macro*/ static moduledata_t test_conf ={ "WOLgenerator", /*module name */ event_handler, /* event handler function*/ "NULL" /* extra data*/ }; /*int dev_open(struct cdev *dev, int oflags, int devtype, struct thread *td) { int err = 0; if (count > 0) return EBUSY; count = 1; return (err); } int dev_close(struct cdev *dev, int fflag, int devtype, struct thread *td) { int err = 0; count = 0; return (err); } int dev_read(struct cdev *dev, struct uio *uio, int ioflag) { int rv = 0; sprintf(flwbuf, "%016d, %9u \n", bytes,sport); rv = uiomove(flwbuf, MIN(uio->uio_resid, strlen(flwbuf)), uio); return rv; }*/ DECLARE_MODULE(WOLgenerator,test_conf,SI_SUB_KLD,SI_ORDER_ANY); /* the name of module, the moduledata_t structure,???,order of loading*/ From owner-freebsd-net@FreeBSD.ORG Sun Sep 12 14:58:43 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AAD2E106564A for ; Sun, 12 Sep 2010 14:58:43 +0000 (UTC) (envelope-from manishv@lineratesystems.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 4BF798FC13 for ; Sun, 12 Sep 2010 14:58:43 +0000 (UTC) Received: by fxm4 with SMTP id 4so3145083fxm.13 for ; Sun, 12 Sep 2010 07:58:42 -0700 (PDT) MIME-Version: 1.0 Received: by 10.223.123.68 with SMTP id o4mr319310far.76.1284302022864; Sun, 12 Sep 2010 07:33:42 -0700 (PDT) Received: by 10.223.112.16 with HTTP; Sun, 12 Sep 2010 07:33:42 -0700 (PDT) Date: Sun, 12 Sep 2010 08:33:42 -0600 Message-ID: From: Manish Vachharajani To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: No ifp->if_ioctl for SIOCAIFADDR and SIOCDIFADDR causes CARP issues on 7.3? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Sep 2010 14:58:43 -0000 I was poking around some of the in_carp.c code and noticed some odd behavior due to the fact that ifp->if_ioctl is not always called on address configuration. It seems that ifp->if_ioctl is not called for SIOCAIFADDR and SIOCDIFADDR, only SIOCSIFADDR under FreeBSD 7.3 release. I confirmed this with a few debug printfs. This results in a few bugs: 1) Improper advertisement even when no IPs are configured Since carp_del_addr is what decrements sc->naddrs, carp will continue to advertise (with the correct HMAC, if there is only one IP, since the carp_hmac_prepare is not called on address deletion) even when there are no IP addresses on the interface. As a result, you can end up with a master system that will not actually serve for the given IP when packets arrive. To cause this, just do ifconfig carp0 create ifconfig carp0 vhid 10 advskew 0 ifconfig carp0 ifconfig -a # confirm that the carp0 interface is MASTER ifconfig carp0 delete tcpdump -i proto carp #notice the advertisements continue ifconfig carp 0 destroy #now the advertisements stops and the physical interface leaves promiscuous mode, confirm via dmesg and above tcpdump Note further that whatever physical interface is associated with the carp device will stay in promiscuous mode until the carp interface is destroyed for the same reason. 2) Improper operation with multiple IPs per CARP interface if IPs are added in the wrong order Carp will work with one virtual ip per carp interface but causes it to function improperly with multiple IPs. For example, it appears that carp will compute the wrong HMAC if IPs are added in different orders on different machines since the hmac is only updated for the first IP address in the group (set with SIOCSADDR by ifconfig). However, carp_hmac_prepare will do the right thing, if only it were called on SIOCAIFADDR and DIFADDR. 3) Not interoperable with multiple IPs (compared to other CARP implementations) Any CARP implementation (such as OpenBSD's which appears to do the right thing) will not interoperate with FreeBSD's (though I haven't confirmed this by booting OpenBSD and trying it) if multiple IPs are configured because FreeBSD only computes the HMAC based on the first IP address that was added, even though carp_hmac_prepare has code to correctly recompute the hmac when new IPs are added in an address independent fashion. It is just that the function is never called since the update would normally be through ifp->if_ioctl. According to the man page for ifnet(9), this is the correct behavior, if_ioctl is only called for SIOCSIFADDR, not AIFADDR and DIFADDR. How should a driver go about listening for these events if not through ifp->if_ioctl? Manish From owner-freebsd-net@FreeBSD.ORG Sun Sep 12 17:52:19 2010 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25626106566C; Sun, 12 Sep 2010 17:52:19 +0000 (UTC) (envelope-from qingli@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id F02AD8FC19; Sun, 12 Sep 2010 17:52:18 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o8CHqIZB099532; Sun, 12 Sep 2010 17:52:18 GMT (envelope-from qingli@freefall.freebsd.org) Received: (from qingli@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o8CHqIPt099528; Sun, 12 Sep 2010 17:52:18 GMT (envelope-from qingli) Date: Sun, 12 Sep 2010 17:52:18 GMT Message-Id: <201009121752.o8CHqIPt099528@freefall.freebsd.org> To: qingli@FreeBSD.org, freebsd-net@FreeBSD.org, qingli@FreeBSD.org From: qingli@FreeBSD.org Cc: Subject: Re: kern/150481: IFA_RTSELF: freebsd 8.1 - ifdown - ifup - loopback route keeps X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Sep 2010 17:52:19 -0000 Synopsis: IFA_RTSELF: freebsd 8.1 - ifdown - ifup - loopback route keeps Responsible-Changed-From-To: freebsd-net->qingli Responsible-Changed-By: qingli Responsible-Changed-When: Sun Sep 12 17:50:30 UTC 2010 Responsible-Changed-Why: Take ownership of this bug. The bug was introduced in 9/2009. http://www.freebsd.org/cgi/query-pr.cgi?pr=150481 From owner-freebsd-net@FreeBSD.ORG Sun Sep 12 20:54:25 2010 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 44739106564A for ; Sun, 12 Sep 2010 20:54:25 +0000 (UTC) (envelope-from tomelite82@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id DAA068FC08 for ; Sun, 12 Sep 2010 20:54:24 +0000 (UTC) Received: by vws7 with SMTP id 7so4717340vws.13 for ; Sun, 12 Sep 2010 13:54:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=DCnGau2hSpjFDMz47LeakxBxcJ0cZMhk/bGeogt04Z0=; b=kQZ/c9uUmO4g2W4AVNBSa8pa0wRk2Cim2eV4Jle0QAvoXnGBFMLXlZLc0I5kVYZj0X SyvtAh3CWGALCoKvvabQXLodmXXHr7hSHt2jhYPPDqvTFfBoko39/peaHRphXFZg8Y9h OVzIiNxLrdibm4taxd3Ft9jQbHhIV6eIv4dbQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=lZB7o4alAQrAKHabYRYa3UsAZTi8QygV7nvcvhBukvHjL3jzU2FrYI5tWejgIz8av7 BYKCMeQhja+popltkh0KPPSardgluScDewza2Tm52DadmCNOTXurhq+TDAUe09t4BSLZ NXHSmmoT8giUKZKleU1iExrwnh5luK+s8hyD0= MIME-Version: 1.0 Received: by 10.220.124.85 with SMTP id t21mr2042617vcr.5.1284323399640; Sun, 12 Sep 2010 13:29:59 -0700 (PDT) Sender: tomelite82@gmail.com Received: by 10.220.42.80 with HTTP; Sun, 12 Sep 2010 13:29:59 -0700 (PDT) In-Reply-To: References: Date: Sun, 12 Sep 2010 13:29:59 -0700 X-Google-Sender-Auth: c3LAjfwr-cLDl9q1IPipQk8gmFo Message-ID: From: Qing Li To: Ingo Flaschberger Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "Li, Qing" , net@freebsd.org Subject: Re: funny ECMP X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Sep 2010 20:54:25 -0000 I have not been able to reproduce the crash based on the steps you outlined in the bug. I do see there is a logic error in the "for" loop right above = the crashing point can trigger a crash. I am looking at your latest patch. -- Qing On Fri, Sep 10, 2010 at 8:05 PM, Ingo Flaschberger wrote: > Dear Li, > > attached latest version of ecmp patch. > > Kind regards, > =A0 =A0 =A0 =A0Ingo Flaschberger From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 10:41:04 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CF6D4106564A for ; Mon, 13 Sep 2010 10:41:04 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 381388FC1A for ; Mon, 13 Sep 2010 10:41:03 +0000 (UTC) Received: (qmail 49160 invoked from network); 13 Sep 2010 10:36:15 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 13 Sep 2010 10:36:15 -0000 Message-ID: <4C8DFFBF.2060905@freebsd.org> Date: Mon, 13 Sep 2010 12:41:03 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: Randall Stewart References: <4C7A7B25.9040300@freebsd.org> <4C7D0089.1020104@freebsd.org> <4C8AA863.4060908@xiplink.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Robert Watson , freebsd-net@freebsd.org, wollman@hergotha.csail.mit.edu, Karim Fodil-Lemelin Subject: Re: Removal of deprecated implied connect for TCP X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 10:41:04 -0000 Based on the feedback I withdraw the proposal to remove implied connect from TCP. Instead I will look at it closer and fix any loose ends that may have come from other changes in the TCP code. Many good points have been raised and I will repeat them here again for the archives: o In FreeBSD most, if not all, protocols support implied connects, removing it from TCP would make it an outlier. o It is being used by at least one product based on FreeBSD. o It can speed up the data sending phase by sending data on the ACK after SYN-ACK. [In RFC1644 it already sent data on the initial SYN but no one is accepting that anymore.] It is important to note, though, that implied connect in TCP is non-standard and no other even remotely popular OS supports it. Thus any applications making use of it are non-portable. -- Andre On 11.09.2010 17:38, Randall Stewart wrote: > All: > > One thing to note.. when you can do an implied connection setup, the > 3-way hand shake has the potential to carry data (don't know if tcp does in FreeBSD) > on the third leg of the 3-way handshake. > > This is one of the reasons SCTP uses this.. since we often will > carry data an the third and even possibly the 4th leg of the handshake (we > have one extra leg). > > Taking this feature out of TCP will make it so we will be like all other > o/s's and the socket semantic will prevent you from doing data on the third leg.. > > ----SYN----> > <----SYN-ACK--- > ----ACK---> > > instead of > > ----SYN--> > <---SYN-ACk-- > ---ACK+DATA--> > > > In the past I have mentioned in classes I teach that TCP is capable of this but the O/S's > of the world do not allow this later behavior.. > > Just thoughts and ramblings ;-) > > R > > > On Sep 10, 2010, at 2:51 PM, Karim Fodil-Lemelin wrote: > >> On 31/08/2010 5:32 PM, Robert Watson wrote: >>> >>> On Tue, 31 Aug 2010, Andre Oppermann wrote: >>> >>>>> I'm not entirely comfortable with this change, and would like a chance to cogitate on it a bit >>>>> more. While I'm not aware of any applications depending on the semantic for TCP, I know that we >>>>> do use it for UNIX domain sockets. >>>> >>>> I don't have any plans to remove the implied connect support from the socket layer or other >>>> protocols, only from TCP. >>> >>> Right -- the implicit question is: why should TCP be the only stream protocol in our stack *not* >>> to support implied connection, when we plan to continue to support it for all other protocols? >>> >>>> For deprecating this part of the TCP API there is no documentation to the implied connect in >>>> tcp(4). In sendto(2) it doesn't differentiate between protocols and simply says: "... sendto() >>>> and sendmsg() may be used at any time." For MSG_EOF it says that is only supported for >>>> SOCK_STREAM sockets in the PF_INET protocol family. These sentences have to be corrected. >>> >>> In general, deprecating is taken to mean providing significant and explicit advance warning of >>> removal -- for example, updating the 8.x man page to point out that the feature is deprecated and >>> it will not appear in future releases of FreeBSD. >>> >>> Robert >>> >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> Hi, >> >> For what its worth, we at Xiphos (now XipLink), are still using sendto and T/TCP and is one of the >> reasons we've chosen FreeBSD more then 10 years ago! >> >> Best regards, >> >> Karim. >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > ------------------------------ > Randall Stewart > 803-317-4952 (cell) > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 10:54:41 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2678C1065673 for ; Mon, 13 Sep 2010 10:54:41 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 875138FC20 for ; Mon, 13 Sep 2010 10:54:40 +0000 (UTC) Received: (qmail 49253 invoked from network); 13 Sep 2010 10:49:51 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 13 Sep 2010 10:49:51 -0000 Message-ID: <4C8E02EF.6030600@freebsd.org> Date: Mon, 13 Sep 2010 12:54:39 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: Randall Stewart References: <4C7A7B25.9040300@freebsd.org> <4C7D0089.1020104@freebsd.org> <4C8AA863.4060908@xiplink.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Robert Watson , Karim Fodil-Lemelin , freebsd-net@freebsd.org Subject: Re: Removal of deprecated implied connect for TCP X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 10:54:41 -0000 On 11.09.2010 17:38, Randall Stewart wrote: > All: > > One thing to note.. when you can do an implied connection setup, the > 3-way hand shake has the potential to carry data (don't know if tcp does in FreeBSD) > on the third leg of the 3-way handshake. > > This is one of the reasons SCTP uses this.. since we often will > carry data an the third and even possibly the 4th leg of the handshake (we > have one extra leg). > > Taking this feature out of TCP will make it so we will be like all other > o/s's and the socket semantic will prevent you from doing data on the third leg.. > > ----SYN----> > <----SYN-ACK--- > ----ACK---> > > instead of > > ----SYN--> > <---SYN-ACk-- > ---ACK+DATA--> > > > In the past I have mentioned in classes I teach that TCP is capable of this but the O/S's > of the world do not allow this later behavior.. The savings in TCP for the case you describe here are not that great. With piggy-backing data on the third leg you save one (small) ACK packet and one round-trip to userspace for the application to start sending data. There is no need to wait for a full network round trip time to start sending data. The real savings from implied connected with RFC1644 came from sending data together with initial SYN. The receiving side would either queue the data until the 3WHS was complete, or upon later invocations directly create a socket upon SYN. The trouble with the way of doing it in RFC1644 was the very weak protection against very simple DoS attacks. Only a connection count variable was used to prevent fake SYN's from opening new sockets. This plus the required socket layer changes (the implied connect) caused a quick halt if any further RFC1644 adoption. -- Andre From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 11:06:59 2010 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B64841065696 for ; Mon, 13 Sep 2010 11:06:59 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id A2D128FC08 for ; Mon, 13 Sep 2010 11:06:59 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o8DB6xWd001949 for ; Mon, 13 Sep 2010 11:06:59 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o8DB6wKi001947 for freebsd-net@FreeBSD.org; Mon, 13 Sep 2010 11:06:59 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 13 Sep 2010 11:06:59 GMT Message-Id: <201009131106.o8DB6wKi001947@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 11:06:59 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/150257 net [msk] watchdog timeout o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o kern/150247 net [patch] [ixgbe] Version in -current won't build on 7.x o bin/150224 net ppp does not reassign static IP after kill -KILL comma o kern/150148 net [ath] Atheros 5424/2424 - AR2425 stopped working with o kern/150052 net wi(4) driver does not work with wlan(4) driver for Luc f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149937 net [ipfilter] [patch] kernel panic in ipfilter IP fragmen o kern/149804 net [icmp] [panic] ICMP redirect on causes "panic: rtqkill o kern/149786 net [bwn] bwn on Dell Inspiron 1150: connections stall o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149539 net [ath] atheros ar9287 is not supported by ath_hal o kern/149516 net [ath] ath(4) hostap with fake MAC/BSSID results in sta o kern/149373 net [realtek/atheros]: None of my network card working o kern/149307 net [ath] Doesn't work Atheros 9285 o kern/149306 net [alc] Doesn't work Atheros AR8131 PCIe Gigabit Etherne o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148862 net [panic] page fault while in kernel mode at _mtx_lock_s o kern/148322 net [ath] Triggering atheros wifi beacon misses in hostap o kern/148317 net [ath] FreeBSD 7.x hostap memory leak in net80211 or At o kern/148078 net [ath] wireless networking stops functioning o kern/147985 net [alc] alc network driver + tso ( + vlan ? ) does not w o kern/147894 net [ipsec] IPv6-in-IPv4 does not work inside an ESP-only o kern/147862 net [wpi] Possible bug in the wpi driver. Network Manager o kern/147155 net [ip6] setfb not work with ipv6 o kern/146909 net [rue] rue(4) does not detect OQO model01 network contr o kern/146845 net [libc] close(2) returns error 54 (connection reset by o kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146759 net [cxgb] [patch] cxgb panic calling cxgb_set_lro() witho o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146517 net [ath] [wlan] device timeouts for ath wlan device on re o kern/146427 net [mwl] Additional virtual access points don't work on m o kern/146426 net [mwl] 802.11n rates not possible on mwl o kern/146425 net [mwl] mwl dropping all packets during and after high u f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146082 net [ng_l2tp] a false invaliant check was performed in ng_ o kern/146037 net [panic] mpd + CoA = kernel panic o bin/145934 net [patch] add count option to netstat(1) o kern/145826 net [ath] Unable to configure adhoc mode on ath0/wlan0 o kern/145825 net [panic] panic: soabort: so_count o kern/145777 net [wpi] Intel 3945ABG driver breaks the connection after o kern/145728 net [lagg] Stops working lagg between two servers. o amd64/145654 net amd64-curent memory leak in kernel o kern/144987 net [wpi] [panic] injecting packets with wlaninject using o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144642 net [rum] [panic] Enabling rum interface causes panic o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 o kern/144572 net [carp] CARP preemption mode traffic partially goes to f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/143939 net [ipfw] [em] ipfw nat and em interface rxcsum problem o kern/143874 net [wpi] Wireless 3945ABG error. wpi0 could not allocate o kern/143868 net [ath] [patch] [request] allow Atheros watchdog timeout o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi s kern/143666 net [ip6] [request] PMTU black hole detection not implemen o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143595 net [wpi] [panic] Creating virtual interface over wpi0 in o kern/143593 net [ipsec] When using IPSec, tcpdump doesn't show outgoin o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143208 net [ipsec] [gif] IPSec over gif interface not working o conf/143079 net hostapd(8) startup missing multi wlan functionality o kern/143074 net [wi]: wi driver triggers panic o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142907 net [wpi] if_wpi unstable on ibm/lenovo x60 -- suspect fir o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 o kern/141777 net [rum] [patch] Support usbdevs / rum(4) for Buffalo WLI f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/141023 net [carp] CARP arp replays with wrong src mac o kern/140796 net [ath] [panic] privileged instruction fault o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph o kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140564 net [wpi] Problem with Intel(R) PRO/Wireless 3945ABG o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140245 net [ath] [panic] Kernel panic during network activity on o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139565 net [ipfilter] ipfilter ioctl SIOCDELST broken o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL o kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/139079 net [wpi] Failure to attach wpi(4) o kern/139058 net [ipfilter] mbuf cluster leak on FreeBSD 7.2 o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138739 net [wpi] wpi(4) does not work very well under 8.0-BETA4 o amd64/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138620 net [lagg] [patch] lagg port bpf-writes blocked o kern/138427 net [wpi] [panic] Kernel panic after trying set monitor wl o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r o kern/138177 net [ipfilter] FreeBSD crashing repeatedly in ip_nat.c:257 o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 o bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed o kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o kern/137775 net [netgraph] [patch] Add XMIT_FAILOVER to ng_one2many o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137592 net [ath] panic - 7-STABLE (Aug 7, 2009 UTC) crashes on ne o bin/137484 net [patch] Integer overflow in wpa_supplicant(8) base64 e o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o bin/136994 net [patch] ifconfig(8) print carp mac address o kern/136943 net [wpi] [lor] wpi0_com_lock / wpi0 o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136836 net [ath] atheros card stops functioning after about 12 ho o bin/136661 net [patch] ndp(8) ignores -f option o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/136426 net [panic] spawning several dhclients in parallel panics o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134931 net [route] Route messages sent to all socket listeners re o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134168 net [ral] ral driver problem on RT2525 2.4GHz transceiver o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133902 net [tun] Killing tun0 iface ssh tunnel causes Panic Strin o kern/133736 net [udp] ip_id not protected ... o kern/133613 net [wpi] [panic] kernel panic in wpi(4) o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre o kern/133218 net [carp] [hang] use of carp(4) causes system to freeze f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o kern/132885 net [wlan] 802.1x broken after SVN rev 189592 o conf/132851 net [patch] rc.conf(5): allow to setfib(1) for service run o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132722 net [ath] Wifi ath0 associates fine with AP, but DHCP or I o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132554 net [ipl] There is no ippool start script/ipfilter magic t o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/132285 net [carp] alias gives incorrect hash in dmesg o kern/132277 net [crypto] [ipsec] poor performance using cryptodevice f o kern/132107 net [carp] carp(4) advskew setting ignored when carp IP us o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131567 net [socket] [patch] Update for regression/sockets/unix_cm o kern/131549 net ifconfig(8) can't clear 'monitor' mode on the wireless o bin/131365 net route(8): route add changes interpretation of network o kern/131153 net [iwi] iwi doesn't see a wireless network f kern/131087 net [ipw] [panic] ipw / iwi - no sent/received packets; iw f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o conf/130555 net [rc.d] [patch] No good way to set ipfilter variables a o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129750 net [ath] Atheros AR5006 exits on "cannot map register spa f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow o kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o bin/128954 net ifconfig(8) deletes valid routes o kern/128917 net [wpi] [panic] if_wpi and wpa+tkip causing kernel panic o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o conf/128334 net [request] use wpa_cli in the "WPA DHCP" situation o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/127102 net [wpi] Intel 3945ABG low throughput o kern/127057 net [udp] Unable to send UDP packet via IPv6 socket to IPv o kern/127050 net [carp] ipv6 does not work on carp interfaces [regressi o kern/126945 net [carp] CARP interface destruction with ifconfig destro o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126714 net [carp] CARP interface renaming makes system no longer o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126475 net [ath] [panic] ath pcmcia card inevitably panics under o kern/126339 net [ipw] ipw driver drops the connection o kern/126214 net [ath] txpower problem with Atheros wifi card o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125816 net [carp] [if_bridge] carp stuck in init when using bridg o kern/125721 net [ath] Terrible throughput/high ping latency with Ubiqu o kern/125617 net [ath] [panic] ath(4) related panic f kern/125502 net [ral] ifconfig ral0 scan produces no output unless in o kern/125501 net [ath] atheros cardbus driver hangs f kern/125332 net [ath] [panic] crash under any non-tiny networking unde o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124767 net [iwi] Wireless connection using iwi0 driver (Intel 220 o kern/124753 net [ieee80211] net80211 discards power-save queue packets o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. p kern/123961 net [vr] [patch] Allow vr interface to handle vlans o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123796 net [ipf] FreeBSD 6.1+VPN+ipnat+ipf: port mapping does not o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o kern/123429 net [nfe] [hang] "ifconfig nfe up" causes a hard system lo o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123256 net [wpi] panic: blockable sleep lock with wpi(4) o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122697 net [ath] Atheros card is not well supported o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw o kern/121872 net [wpi] driver fails to attach on a fujitsu-siemens s711 s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121706 net [netinet] [patch] "rtfree: 0xc4383870 has 1 refs" emit o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o bin/121359 net [patch] [security] ppp(8): fix local stack overflow in o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption p docs/120945 net [patch] ip6(4) man page lacks documentation for TCLASS o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o kern/120130 net [carp] [panic] carp causes kernel panics in any conste o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module s kern/117717 net [panic] Kernel panic with Bittorrent client. o kern/117448 net [carp] 6.2 kernel crash [regression] o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o kern/117271 net [tap] OpenVPN TAP uses 99% CPU on releng_6 when if_tap o kern/116747 net [ndis] FreeBSD 7.0-CURRENT crash with Dell TrueMobile o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111457 net [ral] ral(4) freeze o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o kern/109308 net [pppd] [panic] Multiple panics kernel ppp suspected [r o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] o kern/107944 net [wi] [patch] Forget to unlock mutex-locks f kern/107279 net [ath] [panic] ath_start: attempted use of a free mbuf! o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106438 net [ipf] ipfilter: keep state does not seem to allow repl o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] f kern/105348 net [ath] ath device stopps TX o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/101948 net [ipf] [panic] Kernel Panic Trap No 12 Page Fault - cau o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98978 net [ipf] [patch] ipfilter drops OOW packets under 6.1-Rel o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o bin/98218 net wpa_supplicant(8) blacklist not working f bin/97392 net ppp(8) hangs instead terminating o kern/97306 net [netgraph] NG_L2TP locks after connection with failed f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93886 net [ath] Atheros/D-Link DWL-G650 long delay to associate f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 s kern/91777 net [ipf] [patch] wrong behaviour with skip rule inside an o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging s kern/90086 net [hang] 5.4p8 on supermicro P8SCT hangs during boot if o kern/87521 net [ipf] [panic] using ipfilter "auth" keyword leads to k o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge s kern/86920 net [ndis] ifconfig: SIOCS80211: Invalid argument [regress o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/86103 net [ipf] Illegal NAT Traversal in IPFilter o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/79895 net [ipf] 5.4-RC2 breaks ipfilter NAT when using netgraph o bin/79228 net [patch] extend arp(8) to be able to create blackhole r o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/77273 net [ipf] ipfilter breaks ipv6 statefull filtering on 5.3 s kern/77195 net [ipf] [patch] ipfilter ioctl SIOCGNATL does not match o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/70904 net [ipf] ipfilter ipnat problem with h323 proxy support o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message o kern/65616 net IPSEC can't detunnel GRE packets after real ESP encryp s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong o kern/54383 net [nfs] [patch] NFS root configurations without dynamic s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr s kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/34665 net [ipf] [hang] ipfilter rcmd proxy "hangs". o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/27474 net [ipf] [ppp] Interactive use of user PPP and ipfilter c o conf/23063 net [arp] [patch] for static ARP tables in rc.network 366 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 12:00:32 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 50A2310656C4 for ; Mon, 13 Sep 2010 12:00:32 +0000 (UTC) (envelope-from oppermann@networx.ch) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id B82348FC26 for ; Mon, 13 Sep 2010 12:00:31 +0000 (UTC) Received: (qmail 49519 invoked from network); 13 Sep 2010 11:29:01 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 13 Sep 2010 11:29:01 -0000 Message-ID: <4C8E0C1E.2020707@networx.ch> Date: Mon, 13 Sep 2010 13:33:50 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-current@freebsd.org, freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: TCP loopback socket fusing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 12:00:32 -0000 When a TCP connection via loopback back to localhost is made the whole send, segmentation and receive path (with larger packets though) is still executed. This has some considerable overhead. To short-circuit the send and receive sockets on localhost TCP connections I've made a proof-of-concept patch that directly places the data in the other side's socket buffer without doing any packetization and other protocol overhead (like UNIX domain sockets). The connections setup (SYN, SYN-ACK, ACK) and shutdown are still handled by normal TCP segments via loopback so that firewalling stills works. The actual payload data during the session won't be seen and the sequence numbers don't move other than for SYN and FIN. The sequence are remain valid though. Obviously tcpdump won't see any data transfers either if the connection has fused sockets. Preliminary testing (with WITNESS and INVARIANTS enabled) has shown stable operation and a rough doubling of the throughput on loopback connections. I've tested most socket teardown cases and it behaves fine. I'm not entirely sure I've got all possible path's but the way it is integrated should properly defuse the sockets in all situations. Testers and feedback wanted: http://people.freebsd.org/~andre/tcp_loopfuse-20100913.diff -- Andre From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 12:22:24 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 24D17106564A; Mon, 13 Sep 2010 12:22:24 +0000 (UTC) (envelope-from lars.eggert@Nokia.com) Received: from mgw-sa01.nokia.com (smtp.nokia.com [147.243.1.47]) by mx1.freebsd.org (Postfix) with ESMTP id A400B8FC1A; Mon, 13 Sep 2010 12:22:23 +0000 (UTC) Received: from mail.fit.nokia.com (esdhcp030222.research.nokia.com [172.21.30.222]) by mgw-sa01.nokia.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id o8DBkKc9028124 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 13 Sep 2010 14:46:21 +0300 X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.96.2 at fit.nokia.com Mime-Version: 1.0 (Apple Message framework v1081) Content-Type: text/plain; charset=us-ascii From: Lars Eggert In-Reply-To: <4C7A7B25.9040300@freebsd.org> Date: Mon, 13 Sep 2010 12:46:08 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <5F3C3872-BC86-46A8-9F6F-4EC980E0F98A@nokia.com> References: <4C7A7B25.9040300@freebsd.org> To: Andre Oppermann X-Mailer: Apple Mail (2.1081) X-Spam-Status: No, score=-100.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, MISSING_SUBJECT,USER_IN_WHITELIST autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on fit.nokia.com X-Nokia-AV: Clean Cc: "freebsd-net@freebsd.org" Subject: Re: Removal of deprecated implied connect for TCP X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 12:22:24 -0000 Hi, On 2010-8-29, at 16:22, Andre Oppermann wrote: > T/TCP was ill-defined and had major security issues and never gained > any support. It has been defunct in FreeBSD and most code has been > removed about 6 years ago. we're also about to declare the T/TCP RFCs Historic. See = http://tools.ietf.org/html/draft-eggert-tcpm-historicize (which is a = work item in the TCPM working group despite not being a draft-ietf-... = at the moment.) Lars= From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 12:54:36 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2744D106564A for ; Mon, 13 Sep 2010 12:54:36 +0000 (UTC) (envelope-from oppermann@networx.ch) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 90A868FC25 for ; Mon, 13 Sep 2010 12:54:35 +0000 (UTC) Received: (qmail 50299 invoked from network); 13 Sep 2010 12:49:46 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 13 Sep 2010 12:49:46 -0000 Message-ID: <4C8E1F0B.5090406@networx.ch> Date: Mon, 13 Sep 2010 14:54:35 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: Poul-Henning Kamp References: <47011.1284381913@critter.freebsd.dk> In-Reply-To: <47011.1284381913@critter.freebsd.dk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org Subject: Re: TCP loopback socket fusing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 12:54:36 -0000 On 13.09.2010 14:45, Poul-Henning Kamp wrote: > In message<4C8E0C1E.2020707@networx.ch>, Andre Oppermann writes: > >> To short-circuit the send and receive sockets on localhost TCP connections >> I've made a proof-of-concept patch that directly places the data in the >> other side's socket buffer without doing any packetization and other protocol >> overhead [...] > > Can we keep the option (sysctl ?) of doing the full packet thing, it is > a very convenient debugging tool... Yes, an appropriate sysctl is already contained in the patch (w/o man page documentation yet). -- Andre From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 13:03:55 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5E46D1065679 for ; Mon, 13 Sep 2010 13:03:55 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 1F06C8FC13 for ; Mon, 13 Sep 2010 13:03:54 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id A56A73F592; Mon, 13 Sep 2010 12:45:13 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.4/8.14.4) with ESMTP id o8DCjDEZ047012; Mon, 13 Sep 2010 12:45:13 GMT (envelope-from phk@critter.freebsd.dk) To: Andre Oppermann From: "Poul-Henning Kamp" In-Reply-To: Your message of "Mon, 13 Sep 2010 13:33:50 +0200." <4C8E0C1E.2020707@networx.ch> Date: Mon, 13 Sep 2010 12:45:13 +0000 Message-ID: <47011.1284381913@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org Subject: Re: TCP loopback socket fusing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 13:03:55 -0000 In message <4C8E0C1E.2020707@networx.ch>, Andre Oppermann writes: >To short-circuit the send and receive sockets on localhost TCP connections >I've made a proof-of-concept patch that directly places the data in the >other side's socket buffer without doing any packetization and other protocol >overhead [...] Can we keep the option (sysctl ?) of doing the full packet thing, it is a very convenient debugging tool... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 13:18:06 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4449E1065673 for ; Mon, 13 Sep 2010 13:18:06 +0000 (UTC) (envelope-from marcosvbuzo@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 009058FC17 for ; Mon, 13 Sep 2010 13:18:05 +0000 (UTC) Received: by ywt2 with SMTP id 2so2319110ywt.13 for ; Mon, 13 Sep 2010 06:18:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=TPdqgFoIkYrMbCev5n24CLcdSvRlc306pLdrKZz4ji4=; b=iLt1eOz0DstvLi6G+uBCXX0BJ9dvz9t+d62k1a4QjA7VOJXn/gdqK7Z7YVsRhO+5sL Nakl/wKnZPdkL4nD67dkeClyyKDW2DVJfbFEugzB7IYhLenxZS68hWRCOWCP8r9j4jV2 JRuDqaFtbq5iFfDJ16sYz/UqhqqP2/7gX3Bkw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=krTkuqSJ3GUBCpALsfoUoBUCSUcZrC+O9Rjts1iR5g+SG1BAYiiLgdxy0Gf6JSdoOF xyuqIERRAJ6dKs9JLuoeBK6n1Z5TXXi7NcCZwxn88AjePosazfnrOSHioMtRO0otGvSX or2D3aZ33ecJ/KLmI1MVQdiUMU3mEL6xuty8Q= MIME-Version: 1.0 Received: by 10.151.69.21 with SMTP id w21mr419318ybk.428.1284383885381; Mon, 13 Sep 2010 06:18:05 -0700 (PDT) Received: by 10.231.205.193 with HTTP; Mon, 13 Sep 2010 06:18:04 -0700 (PDT) Date: Mon, 13 Sep 2010 10:18:04 -0300 Message-ID: From: =?ISO-8859-1?Q?Marcos_Vin=EDcius_Buzo?= To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: What about net.isr ? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 13:18:06 -0000 Hi all. I have a dual Intel Xeon E5506 box running mpd5, dummynet and pf. Sometimes i get about 500+ pppoe connections to this machine, the network traffic goes to 30mbps and CPU usage hits 100%. I would like to know if netisr would help me using the other processor cores, and where I can get docs about it. My network card is a dual port Broadcom NetXtreme II BCM5709. Thanks in advance From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 14:11:33 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 63A1D10657E6 for ; Mon, 13 Sep 2010 14:11:33 +0000 (UTC) (envelope-from is@rambler-co.ru) Received: from mailrelay1.rambler.ru (mailrelay1.rambler.ru [81.19.66.239]) by mx1.freebsd.org (Postfix) with ESMTP id CE1F88FC0A for ; Mon, 13 Sep 2010 14:11:32 +0000 (UTC) Received: from localhost (sysoev.ru [81.19.68.137]) by mailrelay1.rambler.ru (Postfix) with ESMTP id 0848E192DCC0; Mon, 13 Sep 2010 18:11:30 +0400 (MSD) Date: Mon, 13 Sep 2010 18:11:30 +0400 From: Igor Sysoev To: Pyun YongHyeon Message-ID: <20100913141130.GK10050@rambler-co.ru> References: <20100909102826.GB53812@rambler-co.ru> <20100909201050.GG7203@michelle.cdnetworks.com> <20100910033915.GA93982@rambler-co.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <20100910033915.GA93982@rambler-co.ru> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-net@freebsd.org Subject: Re: bge hangs on recent 7.3-STABLE X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 14:11:33 -0000 On Fri, Sep 10, 2010 at 07:39:15AM +0400, Igor Sysoev wrote: > On Thu, Sep 09, 2010 at 01:10:50PM -0700, Pyun YongHyeon wrote: > > > On Thu, Sep 09, 2010 at 02:28:26PM +0400, Igor Sysoev wrote: > > > Hi, > > > > > > I have several hosts running FreeBSD/amd64 7.2-STABLE updated on 11.01.2010 > > > and 25.02.2010. Hosts process about 10K input and 10K output packets/s > > > without issues. One of them, however, is loaded more than others, so it > > > processes 20K/20K packets/s. > > > > > > Recently, I have upgraded one host to 7.3-STABLE, 24.08.2010. > > > Then bge on this host hung two times. I was able to restart it from > > > console using: > > > /etc/rc.d/netif restart bge0 > > > > > > Then I have upgraded the most loaded (20K/20K) host to 7.3-STABLE, 07.09.2010. > > > After reboot bge hung every several seconds. I was able to restart it, > > > but bge hung again after several seconds. > > > > > > Then I have downgraded this host to 7.3-STABLE, 14.08.2010, since there > > > were several if_bge.c commits on 15.08.2010. The same hangs. > > > Then I have downgraded this host to 7.3-STABLE, 17.03.2010, before > > > the first if_bge.c commit after 25.02.2010. Now it runs without hangs. > > > > > > The hosts are amd64 dual core SMP with 4G machines. bge information: > > > > > > bge0@pci0:4:0:0: class=0x020000 card=0x165914e4 chip=0x165914e4 rev=0x11 hdr=0x00 > > > vendor = 'Broadcom Corporation' > > > device = 'NetXtreme Gigabit Ethernet PCI Express (BCM5721)' > > > > > > bge0: mem 0xfe5f0000-0xfe5fffff irq 19 at device 0.0 on pci4 > > > miibus1: on bge0 > > > brgphy0: PHY 1 on miibus1 > > > brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto > > > bge0: Ethernet address: 00:e0:81:5f:6e:8a > > > > > > > Could you show me verbose boot message(bge part only)? > > Also show me the output of "pciconf -lcbv". > > Here is "pciconf -lcbv", I will send the "boot -v" part later. > > bge0@pci0:4:0:0: class=0x020000 card=0x165914e4 chip=0x165914e4 rev=0x11 hdr=0x00 > vendor = 'Broadcom Corporation' > device = 'NetXtreme Gigabit Ethernet PCI Express (BCM5721)' > class = network > subclass = ethernet > bar [10] = type Memory, range 64, base 0xfe5f0000, size 65536, enabled > cap 01[48] = powerspec 2 supports D0 D3 current D0 > cap 03[50] = VPD > cap 05[58] = MSI supports 8 messages, 64 bit > cap 10[d0] = PCI-Express 1 endpoint max data 128(128) link x1(x1) Sorry for delay. Here is "boot -v" part. It is from other host, but the host hungs too: pci4: on pcib4 pci4: domain=0, physical bus=4 found-> vendor=0x14e4, dev=0x1659, revid=0x11 domain=0, bus=4, slot=0, func=0 class=02-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=5 powerspec 2 supports D0 D3 current D0 MSI supports 8 messages, 64 bit map[10]: type Memory, range 64, base 0xfe5f0000, size 16, enabled pcib4: requested memory range 0xfe5f0000-0xfe5fffff: good pcib0: matched entry for 0.13.INTA (src \_SB_.PCI0.APC4:0) pcib0: slot 13 INTA routed to irq 19 via \_SB_.PCI0.APC4 pcib4: slot 0 INTA is routed to irq 19 pci0:4:0:0: bad VPD cksum, remain 14 bge0: mem 0 xfe5f0000-0xfe5fffff irq 19 at device 0.0 on pci4 bge0: Reserved 0x10000 bytes for rid 0x10 type 3 at 0xfe5f0000 bge0: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E miibus1: on bge0 brgphy0: PHY 1 on miibus1 brgphy0: OUI 0x000818, model 0x0018, rev. 0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto bge0: bpf attached bge0: Ethernet address: 00:e0:81:5c:64:85 ioapic0: routing intpin 19 (PCI IRQ 19) to vector 54 bge0: [MPSAFE] bge0: [ITHREAD] -- Igor Sysoev http://sysoev.ru/en/ From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 14:27:18 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 82F74106566C for ; Mon, 13 Sep 2010 14:27:18 +0000 (UTC) (envelope-from is@rambler-co.ru) Received: from mailrelay1.rambler.ru (mailrelay1.rambler.ru [81.19.66.239]) by mx1.freebsd.org (Postfix) with ESMTP id 3DFD28FC1B for ; Mon, 13 Sep 2010 14:27:17 +0000 (UTC) Received: from localhost (sysoev.ru [81.19.68.137]) by mailrelay1.rambler.ru (Postfix) with ESMTP id CFA86192DCD7; Mon, 13 Sep 2010 18:27:10 +0400 (MSD) Date: Mon, 13 Sep 2010 18:27:08 +0400 From: Igor Sysoev To: Pyun YongHyeon Message-ID: <20100913142707.GL10050@rambler-co.ru> References: <20100909102826.GB53812@rambler-co.ru> <20100909201050.GG7203@michelle.cdnetworks.com> <20100909211808.GJ7203@michelle.cdnetworks.com> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <20100909211808.GJ7203@michelle.cdnetworks.com> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-net@freebsd.org Subject: Re: bge hangs on recent 7.3-STABLE X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 14:27:18 -0000 On Thu, Sep 09, 2010 at 02:18:08PM -0700, Pyun YongHyeon wrote: > On Thu, Sep 09, 2010 at 01:10:50PM -0700, Pyun YongHyeon wrote: > > On Thu, Sep 09, 2010 at 02:28:26PM +0400, Igor Sysoev wrote: > > > Hi, > > > > > > I have several hosts running FreeBSD/amd64 7.2-STABLE updated on 11.01.2010 > > > and 25.02.2010. Hosts process about 10K input and 10K output packets/s > > > without issues. One of them, however, is loaded more than others, so it > > > processes 20K/20K packets/s. > > > > > > Recently, I have upgraded one host to 7.3-STABLE, 24.08.2010. > > > Then bge on this host hung two times. I was able to restart it from > > > console using: > > > /etc/rc.d/netif restart bge0 > > > > > > Then I have upgraded the most loaded (20K/20K) host to 7.3-STABLE, 07.09.2010. > > > After reboot bge hung every several seconds. I was able to restart it, > > > but bge hung again after several seconds. > > > > > > Then I have downgraded this host to 7.3-STABLE, 14.08.2010, since there > > > were several if_bge.c commits on 15.08.2010. The same hangs. > > > Then I have downgraded this host to 7.3-STABLE, 17.03.2010, before > > > the first if_bge.c commit after 25.02.2010. Now it runs without hangs. > > > > > > The hosts are amd64 dual core SMP with 4G machines. bge information: > > > > > > bge0@pci0:4:0:0: class=0x020000 card=0x165914e4 chip=0x165914e4 rev=0x11 hdr=0x00 > > > vendor = 'Broadcom Corporation' > > > device = 'NetXtreme Gigabit Ethernet PCI Express (BCM5721)' > > > > > > bge0: mem 0xfe5f0000-0xfe5fffff irq 19 at device 0.0 on pci4 > > > miibus1: on bge0 > > > brgphy0: PHY 1 on miibus1 > > > brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto > > > bge0: Ethernet address: 00:e0:81:5f:6e:8a > > > > > > > Could you show me verbose boot message(bge part only)? > > Also show me the output of "pciconf -lcbv". > > > > Forgot to send a patch. Let me know whether attached patch fixes > the issue or not. > Index: sys/dev/bge/if_bge.c > =================================================================== > --- sys/dev/bge/if_bge.c (revision 212341) > +++ sys/dev/bge/if_bge.c (working copy) > @@ -3386,9 +3386,11 @@ > sc->bge_rx_saved_considx = rx_cons; > bge_writembx(sc, BGE_MBX_RX_CONS0_LO, sc->bge_rx_saved_considx); > if (stdcnt) > - bge_writembx(sc, BGE_MBX_RX_STD_PROD_LO, sc->bge_std); > + bge_writembx(sc, BGE_MBX_RX_STD_PROD_LO, (sc->bge_std + > + BGE_STD_RX_RING_CNT - 1) % BGE_STD_RX_RING_CNT); > if (jumbocnt) > - bge_writembx(sc, BGE_MBX_RX_JUMBO_PROD_LO, sc->bge_jumbo); > + bge_writembx(sc, BGE_MBX_RX_JUMBO_PROD_LO, (sc->bge_jumbo + > + BGE_JUMBO_RX_RING_CNT - 1) % BGE_JUMBO_RX_RING_CNT); > #ifdef notyet > /* > * This register wraps very quickly under heavy packet drops. Thank you, it seems the patch has fixed the bug. BTW, I noticed the same hungs on FreeBSD 8.1, date=2010.09.06.23.59.59 I will apply the patch on all my updated hosts. -- Igor Sysoev http://sysoev.ru/en/ From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 15:04:39 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 947AC1065672; Mon, 13 Sep 2010 15:04:39 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: from eu1sys200aog110.obsmtp.com (eu1sys200aog110.obsmtp.com [207.126.144.129]) by mx1.freebsd.org (Postfix) with SMTP id 126548FC25; Mon, 13 Sep 2010 15:04:37 +0000 (UTC) Received: from source ([63.174.175.251]) by eu1sys200aob110.postini.com ([207.126.147.11]) with SMTP ID DSNKTI49g9JBjfe21ffOiFCziqNGiKg4k0id@postini.com; Mon, 13 Sep 2010 15:04:38 UTC Received: from [172.17.10.53] (unknown [172.17.10.53]) by bbbx3.usdmm.com (Postfix) with ESMTP id 133E3FD01A; Mon, 13 Sep 2010 15:04:34 +0000 (UTC) Message-ID: <4C8E3D79.6090102@tomjudge.com> Date: Mon, 13 Sep 2010 10:04:25 -0500 From: Tom Judge User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.12) Gecko/20100826 Lightning/1.0b1 Thunderbird/3.0.7 MIME-Version: 1.0 To: pyunyh@gmail.com References: <4C894A76.5040200@tomjudge.com> <20100910002439.GO7203@michelle.cdnetworks.com> In-Reply-To: <20100910002439.GO7203@michelle.cdnetworks.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, davidch@broadcom.com, yongari@freebsd.org Subject: Re: bce(4) - com_no_buffers (Again) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 15:04:39 -0000 On 09/09/2010 07:24 PM, Pyun YongHyeon wrote: > On Thu, Sep 09, 2010 at 03:58:30PM -0500, Tom Judge wrote: > >> Hi, >> I am just following up on the thread from March (I think) about this issue. >> >> We are seeing this issue on a number of systems running 7.1. >> >> The systems in question are all Dell: >> >> * R710 R610 R410 >> * PE2950 >> >> The latter do not show the issue as much as the R series systems. >> >> The cards in one of the R610's that I am testing with are: >> >> bce0@pci0:1:0:0: class=0x020000 card=0x02361028 chip=0x163914e4 >> rev=0x20 hdr=0x00 >> vendor = 'Broadcom Corporation' >> device = 'NetXtreme II BCM5709 Gigabit Ethernet' >> class = network >> subclass = ethernet >> >> They are connected to Dell PowerConnect 5424 switches. >> >> uname -a: >> FreeBSD bandor.chi-dc.mintel.ad 7.1-RELEASE-p4 FreeBSD 7.1-RELEASE-p4 >> #3: Wed Sep 8 08:19:03 UTC 2010 >> tj@dev-tj-7-1-amd64.chicago.mintel.ad:/usr/obj/usr/src/sys/MINTELv10 amd64 >> >> We are also using 8192 byte jumbo frames, if_lagg and if_vlan in the >> configuration (the nics are in promisc as we are currently capturing >> netflow data on another vlan for diagnostic purposes. ): >> >> >> >> I have updated the bce driver and the Broadcomm MII driver to the >> version from stable/7 and am still seeing the issue. >> >> This morning I did a test with increasing the RX_PAGES to 8 but the >> system just hung starting the network. The route command got stuck in a >> zone state (Sorry can't remember exactly which). >> >> The real question is, how do we go about increasing the number of RX >> BDs? I guess we have to bump more that just RX_PAGES... >> >> >> The cause for us, from what we can see, is the openldap server sending >> large group search results back to nss_ldap or pam_ldap. When it does >> this it seems to send each of the 600 results in its own TCP segment >> creating a small packet storm (600*~100byte PDU's) at the destination >> host. The kernel then retransmits 2 blocks of 100 results each after >> SACK kicks in for the data that was dropped by the NIC. >> >> >> Thanks in advance >> >> Tom >> >> >> > FW may drop incoming frames when it does not see available RX > buffers. Increasing number of RX buffers slightly reduce the > possibility of dropping frames but it wouldn't completely fix it. > Alternatively driver may tell available RX buffers in the middle > of RX ring processing instead of giving updated buffers at the end > of RX processing. This way FW may see available RX buffers while > driver/upper stack is busy to process received frames. But this may > introduce coherency issues because the RX ring is shared between > host and FW. If FreeBSD has way to sync partial region of a DMA > map, this could be implemented without fear of coherency issue. > Another way to improve RX performance would be switching to > multi-RX queue with RSS but that would require a lot of work and I > had no time to implement it. > Does this mean that these cards are going to perform badly? This is was what I gathered from the previous thread. > BTW, given that you've updated to bce(4)/mii(4) of stable/7, I > wonder why TX/RX flow controls were not kicked in. > The working copy I used for grabbing the upstream source is at r212371. Last changes for the directories in my working copy: sys/dev/bce @ 211388 sys/dev/mii @ 212020 I discovered that flow control was disabled on the switches, so I set it to auto and added a pair of BCE_PRINTF's in the code where it enables and disables flow control and now it gets enabled. Without BCE_JUMBO_HDRSPLIT then we see no errors. With it we see number of errors, however the rate seems to be reduced compaired to the previous version of the driver. Tom -- TJU13-ARIN From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 15:08:34 2010 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 304531065670 for ; Mon, 13 Sep 2010 15:08:34 +0000 (UTC) (envelope-from admin@shtorm.com) Received: from ns.shtorm.com (ns.shtorm.com [195.62.14.3]) by mx1.freebsd.org (Postfix) with ESMTP id E31208FC14 for ; Mon, 13 Sep 2010 15:08:33 +0000 (UTC) Received: from [10.66.6.77] (unknown [10.66.6.77]) by ns.shtorm.com (Postfix) with ESMTP id 8FB3B1A0C33; Mon, 13 Sep 2010 17:48:53 +0300 (EEST) From: Shtorm To: Marcos =?ISO-8859-1?Q?Vin=EDcius?= Buzo In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Date: Mon, 13 Sep 2010 17:52:08 +0300 Message-ID: <1284389528.2196.87.camel@stormi-desktop> Mime-Version: 1.0 X-Mailer: Evolution 2.28.3 Content-Transfer-Encoding: 8bit Cc: freebsd-net Subject: Re: What about net.isr ? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 15:08:34 -0000 On Mon, 2010-09-13 at 10:18 -0300, Marcos Vinícius Buzo wrote: > Hi all. > > I have a dual Intel Xeon E5506 box running mpd5, dummynet and pf. Sometimes > i get about 500+ pppoe connections to this machine, the network traffic goes > to 30mbps and CPU usage hits 100%. I would like to know if netisr would help > me using the other processor cores, and where I can get docs about it. > My network card is a dual port Broadcom NetXtreme II BCM5709. > > Thanks in advance > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" In case of pppoe server netisr does not gave me any benefits, it tries to handle all traffic with one thread. Have no idea why, maybe it was my mistake in setup. Check what your broadcom card can do, if it have msi-x with multiple vectors it is better to setup number of vectors to number of cores (maybe number of cores -1) and set sysctls net.isr.direct=1 and net.isr.direct_force=1. In this case traffic processing will be divided between cpu cores and your router will feel much better. I'm using intel network cards on single xeon 5620 for pppoe + dummynet + nat, box handles up to 70 kpps traffic 800+ connections. Also, megabit/s does not matter for routers, packets/s is a thing that gives real cpu load. From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 17:40:50 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 27EFB106566C; Mon, 13 Sep 2010 17:40:50 +0000 (UTC) (envelope-from mdounin@mdounin.ru) Received: from mdounin.cust.ramtel.ru (mdounin.cust.ramtel.ru [81.19.69.81]) by mx1.freebsd.org (Postfix) with ESMTP id 9B2D98FC13; Mon, 13 Sep 2010 17:40:49 +0000 (UTC) Received: from mdounin.ru (mdounin.cust.ramtel.ru [81.19.69.81]) by mdounin.cust.ramtel.ru (Postfix) with ESMTP id EB49E1701C; Mon, 13 Sep 2010 21:24:53 +0400 (MSD) Date: Mon, 13 Sep 2010 21:24:53 +0400 From: Maxim Dounin To: Andre Oppermann Message-ID: <20100913172453.GG99657@mdounin.ru> References: <20100512124702.GJ2679@rambler-co.ru> <20100713140051.GV99657@mdounin.ru> <4C5BCE48.5070504@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4C5BCE48.5070504@freebsd.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-net@freebsd.org, Igor Sysoev , Lawrence Stewart Subject: Re: net.inet.tcp.slowstart_flightsize in 8-STABLE X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 17:40:50 -0000 Hello! On Fri, Aug 06, 2010 at 10:56:40AM +0200, Andre Oppermann wrote: > On 13.07.2010 16:01, Maxim Dounin wrote: > >On Wed, May 12, 2010 at 04:47:02PM +0400, Igor Sysoev wrote: > > > >>It seems that net.inet.tcp.slowstart_flightsize does not work in 8-STABLE. > >>For a long time I used slowstart_flightsize=2 on FreeBSD 4, 6, and 7 hosts. > >>However, FreeBSD-8 always starts with the single packet. > >>I saw this on different versions of 8-STABLE since 8 Oct 2009 till > >>04 Apr 2010. > > > >Finally I had some time to look into it (sorry for long delay). > > > >1. Slow start isn't used on recent FreeBSD versions for initial snd_cwnd > >calculations as long as you have rfc3390 support switched on (default since > >Jan 06 23:29:46 2004, at least in 7.*). It effectively sets initial > >snd_cwnd to 3*MSS on common networks and shouldn't cause any problems. > >Slowstart_flightsize only affects connection restarts. > > > >2. Due to bug in syncache code (patch below) all accepted connections has > >their snd_cwnd reset to 1*MSS (since r171639, 7.0+ AFAIR). > > > >3. Support for rfc3465 introduced in r187289 uncovered (2) as > >ACK to SYN/ACK no longer causes snd_cwnd increase by MSS (actually, this > >increase shouldn't happen as it's explicitly forbidden by rfc 3390, but > >it's another issue). Snd_cwnd remains really small (1*MSS + 1) and this > >causes really bad interaction with delayed acks on other side. > > > >As a workaround to delayed acks interaction problems you may disable > >rfc3465 by setting net.inet.tcp.rfc3465 to 0. Correct fix would be to apply > >the patch below. > > > >To Andre Oppermann: could you please take a look at the patch and > >commit it if found appropriate? > > I've committed your fix with svn r210666. In a few days I will MFC it back > to the stable branches. Thanks for reporting the bug and a patch for it. Andre, could you please take a look at one more patch as well? Igor reported that it still sees 100ms delays with rfc3465 turned on, and it turns out to be similar issue (setting cwnd to 1*MSS) for hosts found in hostcache. The problem with setting cwnd from hostcache was already reported here: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/92690 http://lists.freebsd.org/pipermail/freebsd-net/2007-July/014780.html As using larger cwnd from hostcache may cause problems on congested links (see second thread) I changed code to only use cached cwnd as an upper bound for cwnd (instead of fixing current code). This is also in-line with what we do on connection restarts. We may later consider re-adding usage of larger cwnd from hostcache. But I believe it should be done carefully and probably behind sysctl, off by default. # HG changeset patch # User Maxim Dounin # Date 1284352618 -14400 # Node ID bbb9fea7978b26b95e96d463238a3acd8bfb5575 # Parent 6aec795c568cf6b9d2fabf8b8b9e25ad75b053d0 Use cwnd from hostcache only as upper bound. Setting initial congestion window from hostcache wasn't working for accepted connection since introduction due to tp->snd_wnd being 0. As a result it was instead limiting cwnd on such connections to 1*MSS. With net.inet.tcp.rfc3465 enabled this results bad interaction with delayed acks and 100ms delays for hosts found in hostcache. Additionally, it's considered unsafe to use initial congestion window larger than thouse specified in RFC3390 as this may cause problems on congested links. RFC5681 says equation from RFC3390 MUST be used as upper bound. Links: http://lists.freebsd.org/pipermail/freebsd-net/2007-July/014780.html http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/92690 diff --git a/netinet/tcp_input.c b/netinet/tcp_input.c --- a/netinet/tcp_input.c +++ b/netinet/tcp_input.c @@ -3332,29 +3332,13 @@ tcp_mss(struct tcpcb *tp, int offer) tp->snd_bandwidth = metrics.rmx_bandwidth; /* - * Set the slow-start flight size depending on whether this - * is a local network or not. + * Set initial congestion window per RFC3390. Alternatively, set + * flight size depending on whether this is a local network or not. * - * Extend this so we cache the cwnd too and retrieve it here. - * Make cwnd even bigger than RFC3390 suggests but only if we - * have previous experience with the remote host. Be careful - * not make cwnd bigger than remote receive window or our own - * send socket buffer. Maybe put some additional upper bound - * on the retrieved cwnd. Should do incremental updates to - * hostcache when cwnd collapses so next connection doesn't - * overloads the path again. - * - * RFC3390 says only do this if SYN or SYN/ACK didn't got lost. - * We currently check only in syncache_socket for that. + * RFC3390 says we MUST limit initial window to one segment if SYN + * or SYN/ACK is lost. We currently check only in syncache_socket() + * for that. */ -#define TCP_METRICS_CWND -#ifdef TCP_METRICS_CWND - if (metrics.rmx_cwnd) - tp->snd_cwnd = max(mss, - min(metrics.rmx_cwnd / 2, - min(tp->snd_wnd, so->so_snd.sb_hiwat))); - else -#endif if (V_tcp_do_rfc3390) tp->snd_cwnd = min(4 * mss, max(2 * mss, 4380)); #ifdef INET6 @@ -3367,6 +3351,10 @@ tcp_mss(struct tcpcb *tp, int offer) else tp->snd_cwnd = mss * V_ss_fltsz; + /* If we have cached cwnd - use it as an upper bound. */ + if (metrics.rmx_cwnd) + tp->snd_cwnd = max(mss, min(tp->snd_cwnd, metrics.rmx_cwnd)); + /* Check the interface for TSO capabilities. */ if (mtuflags & CSUM_TSO) tp->t_flags |= TF_TSO; Maxim Dounin From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 18:05:44 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B2ABD106566B for ; Mon, 13 Sep 2010 18:05:44 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 80B498FC24 for ; Mon, 13 Sep 2010 18:05:44 +0000 (UTC) Received: by pzk7 with SMTP id 7so2608905pzk.13 for ; Mon, 13 Sep 2010 11:05:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=8lS2PhEbZLuxLoC6icH25PtK4xCIRCi06IAlJJTOAR8=; b=vULKBghW4flqj706LvpSmUqqSARmACg5tq0+H9dffF1kEgxxUpVWhsx+4BbNiuhwwY u1HAyqFGyuAk3Pp6Ip5kzfLn3T3by1IlUjLp0vd2kVoELxqc6xy4JAB0ARcepi+xR+2v NzMiDUjhmgoZTnsQARUL9fQIha14ITdnMdSC0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=QlnJyaLK9rFYK8EHmoND8mt5EEUOn01bUa7Q1VdfCasrTcEoF6VoGXD/Lfmw8GxX6m Fm2YG+LrQDCS9/quAsvZhoY6sDB02Y16ydNay/cK1sRrmsZOJBFa3YfzHMe6T4E87wZY Ey0f/w66YvbuxqrymUC9dhZHKo2hbjSUMf7ro= Received: by 10.114.74.8 with SMTP id w8mr8423waa.27.1284401139842; Mon, 13 Sep 2010 11:05:39 -0700 (PDT) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id d2sm12487083wam.14.2010.09.13.11.05.36 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 13 Sep 2010 11:05:37 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 13 Sep 2010 11:04:47 -0700 From: Pyun YongHyeon Date: Mon, 13 Sep 2010 11:04:47 -0700 To: Igor Sysoev Message-ID: <20100913180447.GA1229@michelle.cdnetworks.com> References: <20100909102826.GB53812@rambler-co.ru> <20100909201050.GG7203@michelle.cdnetworks.com> <20100909211808.GJ7203@michelle.cdnetworks.com> <20100913142707.GL10050@rambler-co.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100913142707.GL10050@rambler-co.ru> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org Subject: Re: bge hangs on recent 7.3-STABLE X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 18:05:44 -0000 On Mon, Sep 13, 2010 at 06:27:08PM +0400, Igor Sysoev wrote: > On Thu, Sep 09, 2010 at 02:18:08PM -0700, Pyun YongHyeon wrote: > > > On Thu, Sep 09, 2010 at 01:10:50PM -0700, Pyun YongHyeon wrote: > > > On Thu, Sep 09, 2010 at 02:28:26PM +0400, Igor Sysoev wrote: > > > > Hi, > > > > > > > > I have several hosts running FreeBSD/amd64 7.2-STABLE updated on 11.01.2010 > > > > and 25.02.2010. Hosts process about 10K input and 10K output packets/s > > > > without issues. One of them, however, is loaded more than others, so it > > > > processes 20K/20K packets/s. > > > > > > > > Recently, I have upgraded one host to 7.3-STABLE, 24.08.2010. > > > > Then bge on this host hung two times. I was able to restart it from > > > > console using: > > > > /etc/rc.d/netif restart bge0 > > > > > > > > Then I have upgraded the most loaded (20K/20K) host to 7.3-STABLE, 07.09.2010. > > > > After reboot bge hung every several seconds. I was able to restart it, > > > > but bge hung again after several seconds. > > > > > > > > Then I have downgraded this host to 7.3-STABLE, 14.08.2010, since there > > > > were several if_bge.c commits on 15.08.2010. The same hangs. > > > > Then I have downgraded this host to 7.3-STABLE, 17.03.2010, before > > > > the first if_bge.c commit after 25.02.2010. Now it runs without hangs. > > > > > > > > The hosts are amd64 dual core SMP with 4G machines. bge information: > > > > > > > > bge0@pci0:4:0:0: class=0x020000 card=0x165914e4 chip=0x165914e4 rev=0x11 hdr=0x00 > > > > vendor = 'Broadcom Corporation' > > > > device = 'NetXtreme Gigabit Ethernet PCI Express (BCM5721)' > > > > > > > > bge0: mem 0xfe5f0000-0xfe5fffff irq 19 at device 0.0 on pci4 > > > > miibus1: on bge0 > > > > brgphy0: PHY 1 on miibus1 > > > > brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto > > > > bge0: Ethernet address: 00:e0:81:5f:6e:8a > > > > > > > > > > Could you show me verbose boot message(bge part only)? > > > Also show me the output of "pciconf -lcbv". > > > > > > > Forgot to send a patch. Let me know whether attached patch fixes > > the issue or not. > > > Index: sys/dev/bge/if_bge.c > > =================================================================== > > --- sys/dev/bge/if_bge.c (revision 212341) > > +++ sys/dev/bge/if_bge.c (working copy) > > @@ -3386,9 +3386,11 @@ > > sc->bge_rx_saved_considx = rx_cons; > > bge_writembx(sc, BGE_MBX_RX_CONS0_LO, sc->bge_rx_saved_considx); > > if (stdcnt) > > - bge_writembx(sc, BGE_MBX_RX_STD_PROD_LO, sc->bge_std); > > + bge_writembx(sc, BGE_MBX_RX_STD_PROD_LO, (sc->bge_std + > > + BGE_STD_RX_RING_CNT - 1) % BGE_STD_RX_RING_CNT); > > if (jumbocnt) > > - bge_writembx(sc, BGE_MBX_RX_JUMBO_PROD_LO, sc->bge_jumbo); > > + bge_writembx(sc, BGE_MBX_RX_JUMBO_PROD_LO, (sc->bge_jumbo + > > + BGE_JUMBO_RX_RING_CNT - 1) % BGE_JUMBO_RX_RING_CNT); > > #ifdef notyet > > /* > > * This register wraps very quickly under heavy packet drops. > > Thank you, it seems the patch has fixed the bug. > BTW, I noticed the same hungs on FreeBSD 8.1, date=2010.09.06.23.59.59 > I will apply the patch on all my updated hosts. > Thanks for testing. I'm afraid bge(4) in HEAD, stable/8 and stable/7(including 8.1-RELEASE and 7.3-RELEASE) may suffer from this issue. Let me know what other hosts work with the patch. From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 18:20:30 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 867CD1065675 for ; Mon, 13 Sep 2010 18:20:30 +0000 (UTC) (envelope-from is@rambler-co.ru) Received: from mailrelay1.rambler.ru (mailrelay1.rambler.ru [81.19.66.239]) by mx1.freebsd.org (Postfix) with ESMTP id 4238E8FC17 for ; Mon, 13 Sep 2010 18:20:30 +0000 (UTC) Received: from localhost (sysoev.ru [81.19.68.137]) by mailrelay1.rambler.ru (Postfix) with ESMTP id 20DA6192DCCB; Mon, 13 Sep 2010 22:20:28 +0400 (MSD) Date: Mon, 13 Sep 2010 22:20:28 +0400 From: Igor Sysoev To: Pyun YongHyeon Message-ID: <20100913182027.GA27979@rambler-co.ru> References: <20100909102826.GB53812@rambler-co.ru> <20100909201050.GG7203@michelle.cdnetworks.com> <20100909211808.GJ7203@michelle.cdnetworks.com> <20100913142707.GL10050@rambler-co.ru> <20100913180447.GA1229@michelle.cdnetworks.com> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <20100913180447.GA1229@michelle.cdnetworks.com> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-net@freebsd.org Subject: Re: bge hangs on recent 7.3-STABLE X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 18:20:30 -0000 On Mon, Sep 13, 2010 at 11:04:47AM -0700, Pyun YongHyeon wrote: > On Mon, Sep 13, 2010 at 06:27:08PM +0400, Igor Sysoev wrote: > > On Thu, Sep 09, 2010 at 02:18:08PM -0700, Pyun YongHyeon wrote: > > > > > On Thu, Sep 09, 2010 at 01:10:50PM -0700, Pyun YongHyeon wrote: > > > > On Thu, Sep 09, 2010 at 02:28:26PM +0400, Igor Sysoev wrote: > > > > > Hi, > > > > > > > > > > I have several hosts running FreeBSD/amd64 7.2-STABLE updated on 11.01.2010 > > > > > and 25.02.2010. Hosts process about 10K input and 10K output packets/s > > > > > without issues. One of them, however, is loaded more than others, so it > > > > > processes 20K/20K packets/s. > > > > > > > > > > Recently, I have upgraded one host to 7.3-STABLE, 24.08.2010. > > > > > Then bge on this host hung two times. I was able to restart it from > > > > > console using: > > > > > /etc/rc.d/netif restart bge0 > > > > > > > > > > Then I have upgraded the most loaded (20K/20K) host to 7.3-STABLE, 07.09.2010. > > > > > After reboot bge hung every several seconds. I was able to restart it, > > > > > but bge hung again after several seconds. > > > > > > > > > > Then I have downgraded this host to 7.3-STABLE, 14.08.2010, since there > > > > > were several if_bge.c commits on 15.08.2010. The same hangs. > > > > > Then I have downgraded this host to 7.3-STABLE, 17.03.2010, before > > > > > the first if_bge.c commit after 25.02.2010. Now it runs without hangs. > > > > > > > > > > The hosts are amd64 dual core SMP with 4G machines. bge information: > > Thank you, it seems the patch has fixed the bug. > > BTW, I noticed the same hungs on FreeBSD 8.1, date=2010.09.06.23.59.59 > > I will apply the patch on all my updated hosts. > > > > Thanks for testing. I'm afraid bge(4) in HEAD, stable/8 and > stable/7(including 8.1-RELEASE and 7.3-RELEASE) may suffer from > this issue. Let me know what other hosts work with the patch. Currently I have patched two hosts only: 7.3, 24.08.2010 and 8.1, 06.09.2010. 7.3 now handles 20K/20K packets/s without issues. One host has been downgraded to 17.03.2010 as I already reported. Other hosts still run 7.x, from January and February 2010. If there not will be hangs I will upgrade other hosts and will patch them. -- Igor Sysoev http://sysoev.ru/en/ From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 18:49:26 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8EED31065675; Mon, 13 Sep 2010 18:49:26 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 5657A8FC1F; Mon, 13 Sep 2010 18:49:26 +0000 (UTC) Received: by pwi8 with SMTP id 8so2656720pwi.13 for ; Mon, 13 Sep 2010 11:49:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=AY7uykmWQIEehY4B60eKZodpuyNGP+AM1z8eA6Vvlt8=; b=vC0OKem09J8U5dxARxljPN1Nc2s6t4vIBwr7zzwF/ZYSZxMuW1QpQP9ETaV96uqc65 mSv5/SywkuJ7wgV73NGSrS39qZDPgAws8v+0IK40kCqUO6gBxvzpMmpxvmLzbPZZToAF FK/EdXEnaeJcTs+eK+n5ZkvUKa9TF5EdgPIm0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=MA8h/zNKPsm/dm3qjRtPfX3N2RYca7XBRTBwxmanRskt8J0I9uLHfDUNvySgd2yhnK 8Pc3Bq8jYN/gZcfHxf0eS1dBNEhM4Mohyiq+Qp6MgGI3mjDYsz+nkJoI6j7/RpfEfbUc U3U2NW8VhPXa5oBolDdHIF1RQc2adSE8Ze7Z0= Received: by 10.114.121.18 with SMTP id t18mr164262wac.136.1284403764860; Mon, 13 Sep 2010 11:49:24 -0700 (PDT) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id d38sm12546387wam.8.2010.09.13.11.49.22 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 13 Sep 2010 11:49:23 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 13 Sep 2010 11:48:33 -0700 From: Pyun YongHyeon Date: Mon, 13 Sep 2010 11:48:33 -0700 To: Tom Judge Message-ID: <20100913184833.GF1229@michelle.cdnetworks.com> References: <4C894A76.5040200@tomjudge.com> <20100910002439.GO7203@michelle.cdnetworks.com> <4C8E3D79.6090102@tomjudge.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4C8E3D79.6090102@tomjudge.com> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org, davidch@broadcom.com, yongari@freebsd.org Subject: Re: bce(4) - com_no_buffers (Again) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 18:49:26 -0000 On Mon, Sep 13, 2010 at 10:04:25AM -0500, Tom Judge wrote: > On 09/09/2010 07:24 PM, Pyun YongHyeon wrote: > > On Thu, Sep 09, 2010 at 03:58:30PM -0500, Tom Judge wrote: > > > >> Hi, > >> I am just following up on the thread from March (I think) about this issue. > >> > >> We are seeing this issue on a number of systems running 7.1. > >> > >> The systems in question are all Dell: > >> > >> * R710 R610 R410 > >> * PE2950 > >> > >> The latter do not show the issue as much as the R series systems. > >> > >> The cards in one of the R610's that I am testing with are: > >> > >> bce0@pci0:1:0:0: class=0x020000 card=0x02361028 chip=0x163914e4 > >> rev=0x20 hdr=0x00 > >> vendor = 'Broadcom Corporation' > >> device = 'NetXtreme II BCM5709 Gigabit Ethernet' > >> class = network > >> subclass = ethernet > >> > >> They are connected to Dell PowerConnect 5424 switches. > >> > >> uname -a: > >> FreeBSD bandor.chi-dc.mintel.ad 7.1-RELEASE-p4 FreeBSD 7.1-RELEASE-p4 > >> #3: Wed Sep 8 08:19:03 UTC 2010 > >> tj@dev-tj-7-1-amd64.chicago.mintel.ad:/usr/obj/usr/src/sys/MINTELv10 amd64 > >> > >> We are also using 8192 byte jumbo frames, if_lagg and if_vlan in the > >> configuration (the nics are in promisc as we are currently capturing > >> netflow data on another vlan for diagnostic purposes. ): > >> > >> > >> > > >> I have updated the bce driver and the Broadcomm MII driver to the > >> version from stable/7 and am still seeing the issue. > >> > >> This morning I did a test with increasing the RX_PAGES to 8 but the > >> system just hung starting the network. The route command got stuck in a > >> zone state (Sorry can't remember exactly which). > >> > >> The real question is, how do we go about increasing the number of RX > >> BDs? I guess we have to bump more that just RX_PAGES... > >> > >> > >> The cause for us, from what we can see, is the openldap server sending > >> large group search results back to nss_ldap or pam_ldap. When it does > >> this it seems to send each of the 600 results in its own TCP segment > >> creating a small packet storm (600*~100byte PDU's) at the destination > >> host. The kernel then retransmits 2 blocks of 100 results each after > >> SACK kicks in for the data that was dropped by the NIC. > >> > >> > >> Thanks in advance > >> > >> Tom > >> > >> > >> > > > FW may drop incoming frames when it does not see available RX > > buffers. Increasing number of RX buffers slightly reduce the > > possibility of dropping frames but it wouldn't completely fix it. > > Alternatively driver may tell available RX buffers in the middle > > of RX ring processing instead of giving updated buffers at the end > > of RX processing. This way FW may see available RX buffers while > > driver/upper stack is busy to process received frames. But this may > > introduce coherency issues because the RX ring is shared between > > host and FW. If FreeBSD has way to sync partial region of a DMA > > map, this could be implemented without fear of coherency issue. > > Another way to improve RX performance would be switching to > > multi-RX queue with RSS but that would require a lot of work and I > > had no time to implement it. > > > > Does this mean that these cards are going to perform badly? This is was > what I gathered from the previous thread. > I mean there are still many rooms to be done in driver for better performance. bce(4) controllers are one of best controllers for servers and driver didn't take full advantage of it. > > BTW, given that you've updated to bce(4)/mii(4) of stable/7, I > > wonder why TX/RX flow controls were not kicked in. > > > > The working copy I used for grabbing the upstream source is at r212371. > > Last changes for the directories in my working copy: > > sys/dev/bce @ 211388 > sys/dev/mii @ 212020 > > > I discovered that flow control was disabled on the switches, so I set it > to auto and added a pair of BCE_PRINTF's in the code where it enables > and disables flow control and now it gets enabled. > Ok. > > Without BCE_JUMBO_HDRSPLIT then we see no errors. With it we see number > of errors, however the rate seems to be reduced compaired to the > previous version of the driver. > It seems there are issues in header splitting and it was disabled by default. Header splitting reduces packet processing overhead in upper layer so it's normal to see better performance with header splitting. From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 19:08:11 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 326BB106566B; Mon, 13 Sep 2010 19:08:11 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: from eu1sys200aog114.obsmtp.com (eu1sys200aog114.obsmtp.com [207.126.144.137]) by mx1.freebsd.org (Postfix) with SMTP id D5DA18FC0A; Mon, 13 Sep 2010 19:08:09 +0000 (UTC) Received: from source ([63.174.175.251]) by eu1sys200aob114.postini.com ([207.126.147.11]) with SMTP ID DSNKTI52l2bh5Gl3RBYpFxIcIKVWVIlB7Nbw@postini.com; Mon, 13 Sep 2010 19:08:10 UTC Received: from [172.17.10.53] (unknown [172.17.10.53]) by bbbx3.usdmm.com (Postfix) with ESMTP id 9852CFD019; Mon, 13 Sep 2010 19:08:06 +0000 (UTC) Message-ID: <4C8E768E.7000003@tomjudge.com> Date: Mon, 13 Sep 2010 14:07:58 -0500 From: Tom Judge User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.12) Gecko/20100826 Lightning/1.0b1 Thunderbird/3.0.7 MIME-Version: 1.0 To: pyunyh@gmail.com References: <4C894A76.5040200@tomjudge.com> <20100910002439.GO7203@michelle.cdnetworks.com> <4C8E3D79.6090102@tomjudge.com> <20100913184833.GF1229@michelle.cdnetworks.com> In-Reply-To: <20100913184833.GF1229@michelle.cdnetworks.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, davidch@broadcom.com, yongari@freebsd.org Subject: Re: bce(4) - com_no_buffers (Again) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 19:08:11 -0000 On 09/13/2010 01:48 PM, Pyun YongHyeon wrote: > On Mon, Sep 13, 2010 at 10:04:25AM -0500, Tom Judge wrote: > >> >> Does this mean that these cards are going to perform badly? This is was >> what I gathered from the previous thread. >> >> > I mean there are still many rooms to be done in driver for better > performance. bce(4) controllers are one of best controllers for > servers and driver didn't take full advantage of it. > > So far our experiences with bce(4) on FreeBSD have been very disappointing. Starting with when Dell switched to bce(4) based NIC's (around the time 6.2 was released and with the introduction of the Power Edge X9XX hardware) we have always had problems with the driver in every release we have used: 6.2, 7.0 and 7.1. Luckily David has been helpful and helped us fix the issues. > >> Without BCE_JUMBO_HDRSPLIT then we see no errors. With it we see number >> of errors, however the rate seems to be reduced compaired to the >> previous version of the driver. >> >> > It seems there are issues in header splitting and it was disabled > by default. Header splitting reduces packet processing overhead in > upper layer so it's normal to see better performance with header > splitting. > The reason that we have had header splitting enabled in the past is that historically there have been issues with memory fragmentation when using 8k jumbo frames (resulting in 9k mbuf's). I have a kernel with the following configuration in testing right now: * Flow control enabled. * Jumbo header splitting turned off. Is there any way that we can fix flow control with jumbo header splitting turned on? Thanks Tom PS. The following test was more than enough to trigger buffer shortages with header splitting on: ( while true; do ldapsearch -h ldap-server1 -b "ou=Some,o=Base" dn; done ) & ( while true; do ldapsearch -h ldap-server1 -b "ou=Some,o=Base" dn; done ) & ( while true; do ldapsearch -h ldap-server1 -b "ou=Some,o=Base" dn; done ) & The search in question returned about 1700 entries. -- TJU13-ARIN From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 19:11:27 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 033DE106564A for ; Mon, 13 Sep 2010 19:11:27 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 2D9B88FC18 for ; Mon, 13 Sep 2010 19:11:25 +0000 (UTC) Received: (qmail 53894 invoked from network); 13 Sep 2010 19:06:33 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 13 Sep 2010 19:06:33 -0000 Message-ID: <4C8E775D.8070202@freebsd.org> Date: Mon, 13 Sep 2010 21:11:25 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: pyunyh@gmail.com References: <4C894A76.5040200@tomjudge.com> <20100910002439.GO7203@michelle.cdnetworks.com> <4C8E3D79.6090102@tomjudge.com> <20100913184833.GF1229@michelle.cdnetworks.com> In-Reply-To: <20100913184833.GF1229@michelle.cdnetworks.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Tom Judge , freebsd-net@freebsd.org, davidch@broadcom.com, yongari@freebsd.org Subject: Re: bce(4) - com_no_buffers (Again) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 19:11:27 -0000 On 13.09.2010 20:48, Pyun YongHyeon wrote: > On Mon, Sep 13, 2010 at 10:04:25AM -0500, Tom Judge wrote: >> Without BCE_JUMBO_HDRSPLIT then we see no errors. With it we see number >> of errors, however the rate seems to be reduced compaired to the >> previous version of the driver. >> > > It seems there are issues in header splitting and it was disabled > by default. Header splitting reduces packet processing overhead in > upper layer so it's normal to see better performance with header > splitting. I'm not sure that header splitting really helps much at least for TCP. The only place where it could make a difference is at socket buffer append time. There the header get 'thrown away'. With header splitting the first mbuf in the chain containing the header can be returned to the free pool. Without header splitting it's just a offset change in the mbuf. IIRC header splitting was introduced with the Tigeon cards which were the first programmable network cards and the first to support putting the header in a different mbuf. Header splitting, in theory, could make a difference with zero copy sockets where the data portion in a separate mbuf is flipped by VM magic into userspace. The trouble is that no driver fully supports the semantics required for page flipping and the zero copy code, if compiled in, is less much less optimized for the non-flipping case than the standard code path. With the many dozen gigabyte per second memory copy bandwidth of current CPU's it remains questionable whether the page-flipping VM magic is actually faster than a plain kernel/userspace copy as in the standard code path. I generally recommend not to use ZERO_COPY_SOCKETS. I suspect in the case of the bce(4) driver the change in header splitting is probably not the cause of the performance difference. -- Andre From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 19:18:41 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09088106564A; Mon, 13 Sep 2010 19:18:41 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: from eu1sys200aog105.obsmtp.com (eu1sys200aog105.obsmtp.com [207.126.144.119]) by mx1.freebsd.org (Postfix) with SMTP id A836A8FC12; Mon, 13 Sep 2010 19:18:38 +0000 (UTC) Received: from source ([63.174.175.251]) by eu1sys200aob105.postini.com ([207.126.147.11]) with SMTP ID DSNKTI55DHGMEIj6SpVR2ib3iyHkDuwz8+7A@postini.com; Mon, 13 Sep 2010 19:18:40 UTC Received: from [172.17.10.53] (unknown [172.17.10.53]) by bbbx3.usdmm.com (Postfix) with ESMTP id 735E8FD01C; Mon, 13 Sep 2010 19:18:36 +0000 (UTC) Message-ID: <4C8E7904.9090004@tomjudge.com> Date: Mon, 13 Sep 2010 14:18:28 -0500 From: Tom Judge User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.12) Gecko/20100826 Lightning/1.0b1 Thunderbird/3.0.7 MIME-Version: 1.0 To: Andre Oppermann References: <4C894A76.5040200@tomjudge.com> <20100910002439.GO7203@michelle.cdnetworks.com> <4C8E3D79.6090102@tomjudge.com> <20100913184833.GF1229@michelle.cdnetworks.com> <4C8E775D.8070202@freebsd.org> In-Reply-To: <4C8E775D.8070202@freebsd.org> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: pyunyh@gmail.com, freebsd-net@freebsd.org, davidch@broadcom.com, yongari@freebsd.org Subject: Re: bce(4) - com_no_buffers (Again) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 19:18:41 -0000 On 09/13/2010 02:11 PM, Andre Oppermann wrote: > On 13.09.2010 20:48, Pyun YongHyeon wrote: >> On Mon, Sep 13, 2010 at 10:04:25AM -0500, Tom Judge wrote: >>> Without BCE_JUMBO_HDRSPLIT then we see no errors. With it we see >>> number >>> of errors, however the rate seems to be reduced compaired to the >>> previous version of the driver. Please note that 'rate' here relates to the rate at which dev.bce.X.com_no_buffers is increasing not to PPS or bandwidth. However the discussion is still interesting. >>> >> >> It seems there are issues in header splitting and it was disabled >> by default. Header splitting reduces packet processing overhead in >> upper layer so it's normal to see better performance with header >> splitting. > > I'm not sure that header splitting really helps much at least for TCP. > The only place where it could make a difference is at socket buffer > append time. There the header get 'thrown away'. With header splitting > the first mbuf in the chain containing the header can be returned to the > free pool. Without header splitting it's just a offset change in the > mbuf. > > IIRC header splitting was introduced with the Tigeon cards which were > the first programmable network cards and the first to support putting > the header in a different mbuf. Header splitting, in theory, could > make a difference with zero copy sockets where the data portion in a > separate mbuf is flipped by VM magic into userspace. The trouble is > that no driver fully supports the semantics required for page flipping > and the zero copy code, if compiled in, is less much less optimized for > the non-flipping case than the standard code path. With the many dozen > gigabyte per second memory copy bandwidth of current CPU's it remains > questionable whether the page-flipping VM magic is actually faster than > a plain kernel/userspace copy as in the standard code path. I generally > recommend not to use ZERO_COPY_SOCKETS. > > I suspect in the case of the bce(4) driver the change in header splitting > is probably not the cause of the performance difference. > -- TJU13-ARIN From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 19:34:15 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DAE26106566B; Mon, 13 Sep 2010 19:34:14 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id A1CBB8FC18; Mon, 13 Sep 2010 19:34:14 +0000 (UTC) Received: by pwi8 with SMTP id 8so2669836pwi.13 for ; Mon, 13 Sep 2010 12:34:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=LATBn5ZRPy2nWQUZPfmnp+Qmp0iw0KjHziaRGc5SRaY=; b=d6jVpjeVhZNNTGyKMkYMdtrVqsjX1CXQiPdmYO7B/DRjBJMd+Vm/4cc+XvBJ7ILsjI 5ceVEQKEKUHFo5pRt8lGsyV9ybk7vTynIBOKa4ILQn9ZjBETD3LGDqAvKPCdJo66nr8w dXGMHGuQtnHZYVx5Ie6krTdT9KayNQ1tOKJic= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=iVQM3zozpv3hqFXmGzL59xN3HfrqgIZTzhE4HMGmlpYudz1aUHzvKyzB2ZgtugkGj0 eGoWi/I/PJ3pH2JWyXH0WDIEeUUWK8k2P+zvC7YC73QPh8ws0dWKLbIU30QsXO+tbZt6 HK+hkcQY1FfA1e8VOo988li2brEfwxZpVDArM= Received: by 10.114.79.10 with SMTP id c10mr494541wab.220.1284406453859; Mon, 13 Sep 2010 12:34:13 -0700 (PDT) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id d39sm12612405wam.4.2010.09.13.12.34.11 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 13 Sep 2010 12:34:12 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 13 Sep 2010 12:33:22 -0700 From: Pyun YongHyeon Date: Mon, 13 Sep 2010 12:33:22 -0700 To: Tom Judge Message-ID: <20100913193322.GG1229@michelle.cdnetworks.com> References: <4C894A76.5040200@tomjudge.com> <20100910002439.GO7203@michelle.cdnetworks.com> <4C8E3D79.6090102@tomjudge.com> <20100913184833.GF1229@michelle.cdnetworks.com> <4C8E768E.7000003@tomjudge.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4C8E768E.7000003@tomjudge.com> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org, davidch@broadcom.com, yongari@freebsd.org Subject: Re: bce(4) - com_no_buffers (Again) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 19:34:15 -0000 On Mon, Sep 13, 2010 at 02:07:58PM -0500, Tom Judge wrote: > On 09/13/2010 01:48 PM, Pyun YongHyeon wrote: > > On Mon, Sep 13, 2010 at 10:04:25AM -0500, Tom Judge wrote: > > > >> > > >> Does this mean that these cards are going to perform badly? This is was > >> what I gathered from the previous thread. > >> > >> > > I mean there are still many rooms to be done in driver for better > > performance. bce(4) controllers are one of best controllers for > > servers and driver didn't take full advantage of it. > > > > > > So far our experiences with bce(4) on FreeBSD have been very > disappointing. Starting with when Dell switched to bce(4) based NIC's > (around the time 6.2 was released and with the introduction of the Power > Edge X9XX hardware) we have always had problems with the driver in every > release we have used: 6.2, 7.0 and 7.1. Luckily David has been helpful > and helped us fix the issues. > > > > > >> Without BCE_JUMBO_HDRSPLIT then we see no errors. With it we see number > >> of errors, however the rate seems to be reduced compaired to the > >> previous version of the driver. > >> > >> > > It seems there are issues in header splitting and it was disabled > > by default. Header splitting reduces packet processing overhead in > > upper layer so it's normal to see better performance with header > > splitting. > > > > The reason that we have had header splitting enabled in the past is that > historically there have been issues with memory fragmentation when using > 8k jumbo frames (resulting in 9k mbuf's). > Yes, if you use jumbo frames, header splitting would help to reduce memory fragmentation as header splitting wouldn't allocate jumbo clusters. > I have a kernel with the following configuration in testing right now: > > * Flow control enabled. > * Jumbo header splitting turned off. > > > Is there any way that we can fix flow control with jumbo header > splitting turned on? > Flow control has nothing to do with header splitting(i.e. flow control is always enabled). > Thanks > > Tom > > PS. The following test was more than enough to trigger buffer shortages > with header splitting on: > > ( while true; do ldapsearch -h ldap-server1 -b "ou=Some,o=Base" dn; done > ) & > ( while true; do ldapsearch -h ldap-server1 -b "ou=Some,o=Base" dn; done > ) & > ( while true; do ldapsearch -h ldap-server1 -b "ou=Some,o=Base" dn; done > ) & > > The search in question returned about 1700 entries. > I can trigger this kind of buffer shortage with benchmark tools. Actually fixing header splitting is on my TODO list as well as other things but I don't know how long it would take. From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 19:40:39 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E300106566C; Mon, 13 Sep 2010 19:40:39 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 0E6908FC1D; Mon, 13 Sep 2010 19:40:38 +0000 (UTC) Received: by pwi8 with SMTP id 8so2671848pwi.13 for ; Mon, 13 Sep 2010 12:40:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=Q2ONQY9VWzkHZhs9qdtLXa33CVmJ53ls0jGrGb4zWUE=; b=lfcDXcZUnsrQcvM+g2o0eF0OcK6GR7TjD3pxxUJgbc9uZ+3VM/v+Ael5RVmQ52u0GS 745Y9yM3/wfx1BJeRqKbYV34BPRYvzV6AbA/+hJqFVPLf+TGbONg9ybN65zhIY3qJcGn Ezl0r/0Jh5gm1r3Sy/yKPd2R5fPv3y9TQ3Mb4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=rrV5T38mgION+CvqYfZme5FkKda4mubWnikIFcRfrNPzZK0MUJC2Oaeeuwb2fFHiW3 Ieo8vsvZ1edOFgmfJmvVKmzvfKIkJCQat2v0W1ZhPvuW9R+9O/2zKR0Qi3ZHv6tMOUks Kf0DBXotSGbeoJeugTXytqLqkPgicrLDtJOQc= Received: by 10.114.131.8 with SMTP id e8mr167380wad.43.1284406838439; Mon, 13 Sep 2010 12:40:38 -0700 (PDT) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 33sm12623187wad.6.2010.09.13.12.40.36 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 13 Sep 2010 12:40:37 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 13 Sep 2010 12:39:47 -0700 From: Pyun YongHyeon Date: Mon, 13 Sep 2010 12:39:47 -0700 To: Andre Oppermann Message-ID: <20100913193947.GH1229@michelle.cdnetworks.com> References: <4C894A76.5040200@tomjudge.com> <20100910002439.GO7203@michelle.cdnetworks.com> <4C8E3D79.6090102@tomjudge.com> <20100913184833.GF1229@michelle.cdnetworks.com> <4C8E775D.8070202@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4C8E775D.8070202@freebsd.org> User-Agent: Mutt/1.4.2.3i Cc: Tom Judge , freebsd-net@freebsd.org, davidch@broadcom.com, yongari@freebsd.org Subject: Re: bce(4) - com_no_buffers (Again) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 19:40:39 -0000 On Mon, Sep 13, 2010 at 09:11:25PM +0200, Andre Oppermann wrote: > On 13.09.2010 20:48, Pyun YongHyeon wrote: > >On Mon, Sep 13, 2010 at 10:04:25AM -0500, Tom Judge wrote: > >>Without BCE_JUMBO_HDRSPLIT then we see no errors. With it we see number > >>of errors, however the rate seems to be reduced compaired to the > >>previous version of the driver. > >> > > > >It seems there are issues in header splitting and it was disabled > >by default. Header splitting reduces packet processing overhead in > >upper layer so it's normal to see better performance with header > >splitting. > > I'm not sure that header splitting really helps much at least for TCP. > The only place where it could make a difference is at socket buffer > append time. There the header get 'thrown away'. With header splitting > the first mbuf in the chain containing the header can be returned to the > free pool. Without header splitting it's just a offset change in the > mbuf. > > IIRC header splitting was introduced with the Tigeon cards which were > the first programmable network cards and the first to support putting > the header in a different mbuf. Header splitting, in theory, could > make a difference with zero copy sockets where the data portion in a > separate mbuf is flipped by VM magic into userspace. The trouble is > that no driver fully supports the semantics required for page flipping > and the zero copy code, if compiled in, is less much less optimized for > the non-flipping case than the standard code path. With the many dozen > gigabyte per second memory copy bandwidth of current CPU's it remains > questionable whether the page-flipping VM magic is actually faster than > a plain kernel/userspace copy as in the standard code path. I generally > recommend not to use ZERO_COPY_SOCKETS. > > I suspect in the case of the bce(4) driver the change in header splitting > is probably not the cause of the performance difference. > I'm under the impression the header splitting in bce(4) is for LRO(opposite of TSO), not for VM magic to enable page flipping tricks. > -- > Andre From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 20:36:57 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 02EAF1065710 for ; Mon, 13 Sep 2010 20:36:57 +0000 (UTC) (envelope-from web1@vcvps1364.vcdns.de) Received: from vcvps1364.vcdns.de (vcvps1364.vcdns.de [88.80.215.7]) by mx1.freebsd.org (Postfix) with ESMTP id BE08B8FC21 for ; Mon, 13 Sep 2010 20:36:56 +0000 (UTC) Received: by vcvps1364.vcdns.de (Postfix, from userid 33) id AD79E10BAE33; Mon, 13 Sep 2010 22:06:40 +0200 (CEST) To: freebsd-net@freebsd.org From: Mme Claire Page MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 8bit Message-Id: <20100913200640.AD79E10BAE33@vcvps1364.vcdns.de> Date: Mon, 13 Sep 2010 22:06:40 +0200 (CEST) Subject: Ramadan heureux mon cher X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: mm.chiwe.conte@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 20:36:57 -0000 I am Mrs Claire Page sending you this mail from my sick bed in the hospital. Please contact my lawyer, Email:(barr_willam_frank@lawyer.com) Je suis Mme Claire Page vous envoie ce mail de mon lit de malade à l'hôpital. S'il vous plaît communiquer avec mon avocat, Email: (barr_willam_frank@lawyer.com) From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 20:38:54 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5BF3F106566C; Mon, 13 Sep 2010 20:38:54 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: from eu1sys200aog108.obsmtp.com (eu1sys200aog108.obsmtp.com [207.126.144.125]) by mx1.freebsd.org (Postfix) with SMTP id DB9A58FC20; Mon, 13 Sep 2010 20:38:52 +0000 (UTC) Received: from source ([63.174.175.251]) by eu1sys200aob108.postini.com ([207.126.147.11]) with SMTP ID DSNKTI6L2nWjw4CgZClUqxXnNBoPuEcW8WwX@postini.com; Mon, 13 Sep 2010 20:38:53 UTC Received: from [172.17.10.53] (unknown [172.17.10.53]) by bbbx3.usdmm.com (Postfix) with ESMTP id B9611FD021; Mon, 13 Sep 2010 20:38:49 +0000 (UTC) Message-ID: <4C8E8BD1.5090007@tomjudge.com> Date: Mon, 13 Sep 2010 15:38:41 -0500 From: Tom Judge User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.12) Gecko/20100826 Lightning/1.0b1 Thunderbird/3.0.7 MIME-Version: 1.0 To: pyunyh@gmail.com References: <4C894A76.5040200@tomjudge.com> <20100910002439.GO7203@michelle.cdnetworks.com> <4C8E3D79.6090102@tomjudge.com> <20100913184833.GF1229@michelle.cdnetworks.com> <4C8E768E.7000003@tomjudge.com> <20100913193322.GG1229@michelle.cdnetworks.com> In-Reply-To: <20100913193322.GG1229@michelle.cdnetworks.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, davidch@broadcom.com, yongari@freebsd.org Subject: Re: bce(4) - com_no_buffers (Again) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 20:38:54 -0000 On 09/13/2010 02:33 PM, Pyun YongHyeon wrote: > On Mon, Sep 13, 2010 at 02:07:58PM -0500, Tom Judge wrote: > >> On 09/13/2010 01:48 PM, Pyun YongHyeon wrote: >> >>> On Mon, Sep 13, 2010 at 10:04:25AM -0500, Tom Judge wrote: >>> >>> >>>> >> >> >>>> Does this mean that these cards are going to perform badly? This is was >>>> what I gathered from the previous thread. >>>> >>>> >>>> >>> I mean there are still many rooms to be done in driver for better >>> performance. bce(4) controllers are one of best controllers for >>> servers and driver didn't take full advantage of it. >>> >>> >>> >> So far our experiences with bce(4) on FreeBSD have been very >> disappointing. Starting with when Dell switched to bce(4) based NIC's >> (around the time 6.2 was released and with the introduction of the Power >> Edge X9XX hardware) we have always had problems with the driver in every >> release we have used: 6.2, 7.0 and 7.1. Luckily David has been helpful >> and helped us fix the issues. >> >> >> >>> >>> >>>> Without BCE_JUMBO_HDRSPLIT then we see no errors. With it we see number >>>> of errors, however the rate seems to be reduced compaired to the >>>> previous version of the driver. >>>> >>>> >>>> >>> It seems there are issues in header splitting and it was disabled >>> by default. Header splitting reduces packet processing overhead in >>> upper layer so it's normal to see better performance with header >>> splitting. >>> >>> >> The reason that we have had header splitting enabled in the past is that >> historically there have been issues with memory fragmentation when using >> 8k jumbo frames (resulting in 9k mbuf's). >> >> > Yes, if you use jumbo frames, header splitting would help to reduce > memory fragmentation as header splitting wouldn't allocate jumbo > clusters. > > Under testing I have yet to see a memory fragmentation issue with this driver. I follow up if/when I find a problem with this again. >> I have a kernel with the following configuration in testing right now: >> >> * Flow control enabled. >> * Jumbo header splitting turned off. >> >> >> Is there any way that we can fix flow control with jumbo header >> splitting turned on? >> >> > Flow control has nothing to do with header splitting(i.e. flow > control is always enabled). > > Sorry let me rephrase that: Is there a way to fix the RX buffer shortage issues (when header splitting is turned on) so that they are guarded by flow control. Maybe change the low watermark for flow control when its enabled? >> Thanks >> >> Tom >> >> PS. The following test was more than enough to trigger buffer shortages >> with header splitting on: >> >> ( while true; do ldapsearch -h ldap-server1 -b "ou=Some,o=Base" dn; done >> ) & >> ( while true; do ldapsearch -h ldap-server1 -b "ou=Some,o=Base" dn; done >> ) & >> ( while true; do ldapsearch -h ldap-server1 -b "ou=Some,o=Base" dn; done >> ) & >> >> The search in question returned about 1700 entries. >> >> > I can trigger this kind of buffer shortage with benchmark tools. > Actually fixing header splitting is on my TODO list as well as > other things but I don't know how long it would take. > Great to here, thanks for all the hard work. Tom -- TJU13-ARIN From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 20:54:40 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6EA2A106564A; Mon, 13 Sep 2010 20:54:40 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 33C608FC13; Mon, 13 Sep 2010 20:54:39 +0000 (UTC) Received: by pwi8 with SMTP id 8so2691867pwi.13 for ; Mon, 13 Sep 2010 13:54:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=6hgzRo4KDQaADKUtPXsEtPxhCTCHc+Zpi2Buf34RVfk=; b=sAOjpbfvmIRiuMpTwLeISbASX7tGKg4GUfsBnMi92RfSyaKUE7sY3OnR/Wn6giabvj wTPuAcxvrh5BZ09/YpCW4GzJCtkTOOtkU4vzX1J7qM9bBO+vBD0+ekKb6VkTHM1SvXO3 zwHKgAmvvl4HZMFRQ/c2XX1aAQUvDOGKJZXho= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=a9NItv23efku/Wt5XwQmk2E6arItsVsgYqPKOJea4n5IT57gI29A9i+rmSKwA0fUZp HXrTjxPCEWm83nQAyIhtwERsbqzFqV2r7ultr99gk/R3HN3zS7MKBqtMvOtTyHgxlRZ5 lyIdu5sKOvIQCXaqLd/2bkIoCxlokJ2l7lzPc= Received: by 10.142.1.37 with SMTP id 37mr205015wfa.242.1284411279643; Mon, 13 Sep 2010 13:54:39 -0700 (PDT) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id n35sm7147458wfa.3.2010.09.13.13.54.36 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 13 Sep 2010 13:54:37 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 13 Sep 2010 13:53:48 -0700 From: Pyun YongHyeon Date: Mon, 13 Sep 2010 13:53:48 -0700 To: Tom Judge Message-ID: <20100913205348.GJ1229@michelle.cdnetworks.com> References: <4C894A76.5040200@tomjudge.com> <20100910002439.GO7203@michelle.cdnetworks.com> <4C8E3D79.6090102@tomjudge.com> <20100913184833.GF1229@michelle.cdnetworks.com> <4C8E768E.7000003@tomjudge.com> <20100913193322.GG1229@michelle.cdnetworks.com> <4C8E8BD1.5090007@tomjudge.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4C8E8BD1.5090007@tomjudge.com> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org, davidch@broadcom.com, yongari@freebsd.org Subject: Re: bce(4) - com_no_buffers (Again) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 20:54:40 -0000 On Mon, Sep 13, 2010 at 03:38:41PM -0500, Tom Judge wrote: > On 09/13/2010 02:33 PM, Pyun YongHyeon wrote: > > On Mon, Sep 13, 2010 at 02:07:58PM -0500, Tom Judge wrote: > > > >> On 09/13/2010 01:48 PM, Pyun YongHyeon wrote: > >> > >>> On Mon, Sep 13, 2010 at 10:04:25AM -0500, Tom Judge wrote: > >>> > >>> > >>>> > >> > >> > >>>> Does this mean that these cards are going to perform badly? This is was > >>>> what I gathered from the previous thread. > >>>> > >>>> > >>>> > >>> I mean there are still many rooms to be done in driver for better > >>> performance. bce(4) controllers are one of best controllers for > >>> servers and driver didn't take full advantage of it. > >>> > >>> > >>> > >> So far our experiences with bce(4) on FreeBSD have been very > >> disappointing. Starting with when Dell switched to bce(4) based NIC's > >> (around the time 6.2 was released and with the introduction of the Power > >> Edge X9XX hardware) we have always had problems with the driver in every > >> release we have used: 6.2, 7.0 and 7.1. Luckily David has been helpful > >> and helped us fix the issues. > >> > >> > >> > >>> > >>> > >>>> Without BCE_JUMBO_HDRSPLIT then we see no errors. With it we see number > >>>> of errors, however the rate seems to be reduced compaired to the > >>>> previous version of the driver. > >>>> > >>>> > >>>> > >>> It seems there are issues in header splitting and it was disabled > >>> by default. Header splitting reduces packet processing overhead in > >>> upper layer so it's normal to see better performance with header > >>> splitting. > >>> > >>> > >> The reason that we have had header splitting enabled in the past is that > >> historically there have been issues with memory fragmentation when using > >> 8k jumbo frames (resulting in 9k mbuf's). > >> > >> > > Yes, if you use jumbo frames, header splitting would help to reduce > > memory fragmentation as header splitting wouldn't allocate jumbo > > clusters. > > > > > > Under testing I have yet to see a memory fragmentation issue with this > driver. I follow up if/when I find a problem with this again. > > >> I have a kernel with the following configuration in testing right now: > >> > >> * Flow control enabled. > >> * Jumbo header splitting turned off. > >> > >> > >> Is there any way that we can fix flow control with jumbo header > >> splitting turned on? > >> > >> > > Flow control has nothing to do with header splitting(i.e. flow > > control is always enabled). > > > > > Sorry let me rephrase that: > > Is there a way to fix the RX buffer shortage issues (when header > splitting is turned on) so that they are guarded by flow control. Maybe > change the low watermark for flow control when its enabled? > I'm not sure how much it would help but try changing RX low watermark. Default value is 32 which seems to be reasonable value. But it's only for 5709/5716 controllers and Linux seems to use different default value. From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 22:08:49 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7B8AA106566C for ; Mon, 13 Sep 2010 22:08:49 +0000 (UTC) (envelope-from dudu@dudu.ro) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3C0398FC14 for ; Mon, 13 Sep 2010 22:08:48 +0000 (UTC) Received: by qwg5 with SMTP id 5so4219646qwg.13 for ; Mon, 13 Sep 2010 15:08:48 -0700 (PDT) Received: by 10.229.1.143 with SMTP id 15mr3374913qcf.287.1284415728156; Mon, 13 Sep 2010 15:08:48 -0700 (PDT) MIME-Version: 1.0 Received: by 10.229.38.83 with HTTP; Mon, 13 Sep 2010 15:08:08 -0700 (PDT) In-Reply-To: <20100913180447.GA1229@michelle.cdnetworks.com> References: <20100909102826.GB53812@rambler-co.ru> <20100909201050.GG7203@michelle.cdnetworks.com> <20100909211808.GJ7203@michelle.cdnetworks.com> <20100913142707.GL10050@rambler-co.ru> <20100913180447.GA1229@michelle.cdnetworks.com> From: Vlad Galu Date: Tue, 14 Sep 2010 01:08:08 +0300 Message-ID: To: pyunyh@gmail.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org, Igor Sysoev Subject: Re: bge hangs on recent 7.3-STABLE X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 22:08:49 -0000 On Mon, Sep 13, 2010 at 9:04 PM, Pyun YongHyeon wrote: > On Mon, Sep 13, 2010 at 06:27:08PM +0400, Igor Sysoev wrote: >> On Thu, Sep 09, 2010 at 02:18:08PM -0700, Pyun YongHyeon wrote: >> >> > On Thu, Sep 09, 2010 at 01:10:50PM -0700, Pyun YongHyeon wrote: >> > > On Thu, Sep 09, 2010 at 02:28:26PM +0400, Igor Sysoev wrote: >> > > > Hi, >> > > > >> > > > I have several hosts running FreeBSD/amd64 7.2-STABLE updated on 1= 1.01.2010 >> > > > and 25.02.2010. Hosts process about 10K input and 10K output packe= ts/s >> > > > without issues. One of them, however, is loaded more than others, = so it >> > > > processes 20K/20K packets/s. >> > > > >> > > > Recently, I have upgraded one host to 7.3-STABLE, 24.08.2010. >> > > > Then bge on this host hung two times. I was able to restart it fro= m >> > > > console using: >> > > > =A0 /etc/rc.d/netif restart bge0 >> > > > >> > > > Then I have upgraded the most loaded (20K/20K) host to 7.3-STABLE,= 07.09.2010. >> > > > After reboot bge hung every several seconds. I was able to restart= it, >> > > > but bge hung again after several seconds. >> > > > >> > > > Then I have downgraded this host to 7.3-STABLE, 14.08.2010, since = there >> > > > were several if_bge.c commits on 15.08.2010. The same hangs. >> > > > Then I have downgraded this host to 7.3-STABLE, 17.03.2010, before >> > > > the first if_bge.c commit after 25.02.2010. Now it runs without ha= ngs. >> > > > >> > > > The hosts are amd64 dual core SMP with 4G machines. bge informatio= n: >> > > > >> > > > bge0@pci0:4:0:0: =A0 =A0 =A0 =A0class=3D0x020000 card=3D0x165914e4= chip=3D0x165914e4 rev=3D0x11 hdr=3D0x00 >> > > > =A0 =A0 vendor =A0 =A0 =3D 'Broadcom Corporation' >> > > > =A0 =A0 device =A0 =A0 =3D 'NetXtreme Gigabit Ethernet PCI Express= (BCM5721)' >> > > > >> > > > bge0: mem 0xfe5f0000-0xfe5fffff irq 19 at device 0.0 on pci4 >> > > > miibus1: on bge0 >> > > > brgphy0: PHY 1 on miibus1 >> > > > brgphy0: =A010baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000ba= seT, 1000baseT-FDX, auto >> > > > bge0: Ethernet address: 00:e0:81:5f:6e:8a >> > > > >> > > >> > > Could you show me verbose boot message(bge part only)? >> > > Also show me the output of "pciconf -lcbv". >> > > >> > >> > Forgot to send a patch. Let me know whether attached patch fixes >> > the issue or not. >> >> > Index: sys/dev/bge/if_bge.c >> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> > --- sys/dev/bge/if_bge.c =A0 =A0(revision 212341) >> > +++ sys/dev/bge/if_bge.c =A0 =A0(working copy) >> > @@ -3386,9 +3386,11 @@ >> > =A0 =A0 sc->bge_rx_saved_considx =3D rx_cons; >> > =A0 =A0 bge_writembx(sc, BGE_MBX_RX_CONS0_LO, sc->bge_rx_saved_considx= ); >> > =A0 =A0 if (stdcnt) >> > - =A0 =A0 =A0 =A0 =A0 bge_writembx(sc, BGE_MBX_RX_STD_PROD_LO, sc->bge= _std); >> > + =A0 =A0 =A0 =A0 =A0 bge_writembx(sc, BGE_MBX_RX_STD_PROD_LO, (sc->bg= e_std + >> > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 BGE_STD_RX_RING_CNT - 1) % BGE_STD_RX_RI= NG_CNT); >> > =A0 =A0 if (jumbocnt) >> > - =A0 =A0 =A0 =A0 =A0 bge_writembx(sc, BGE_MBX_RX_JUMBO_PROD_LO, sc->b= ge_jumbo); >> > + =A0 =A0 =A0 =A0 =A0 bge_writembx(sc, BGE_MBX_RX_JUMBO_PROD_LO, (sc->= bge_jumbo + >> > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 BGE_JUMBO_RX_RING_CNT - 1) % BGE_JUMBO_R= X_RING_CNT); >> > =A0#ifdef notyet >> > =A0 =A0 /* >> > =A0 =A0 =A0* This register wraps very quickly under heavy packet drops= . >> >> Thank you, it seems the patch has fixed the bug. >> BTW, I noticed the same hungs on FreeBSD 8.1, date=3D2010.09.06.23.59.59 >> I will apply the patch on all my updated hosts. >> > > Thanks for testing. I'm afraid bge(4) in HEAD, stable/8 and > stable/7(including 8.1-RELEASE and 7.3-RELEASE) may suffer from > this issue. Let me know what other hosts work with the patch. Hi Pyun, Thanks for the patch. It seems to have fixed the symptom in my case, on a card identical to Igor's, but on board of an IBM eServer 306m. Regards, Vlad --=20 Good, fast & cheap. Pick any two. From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 22:50:22 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2AE821065695; Mon, 13 Sep 2010 22:50:22 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 8E0E48FC17; Mon, 13 Sep 2010 22:50:21 +0000 (UTC) Received: by gwb15 with SMTP id 15so2235393gwb.13 for ; Mon, 13 Sep 2010 15:50:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=FsJFlIvxRlQs6Z/I+Imkzs7xTKaKhS4X83HBizlPGvM=; b=nYUlDmnrV5UqPMN/U85zN2R+uXrOTov6X3hDtkF579IHHDERbrC+g+/x5WBTHLNbW+ NhimHdUkwuY/HFubEADNNyqApWHLxXTpW0F4FICI340c/bahKyBzRAtn5hCCMbUZJ37j bmQXaDCeTXMEQiwXHFZbqu3GDT6M7TYDgZ3fc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=mhpUrbggPOCwQd+/olwQlaDDnWmiWnixFp5G/DH0teT/9S/AxNRdf2if6FAt0fGVp/ Unmjm/o/oTeFPPOuVr5r9dIzpsx2mxUar4JTNYYw5puYVhO7mwtqoXP7sAHgrwpgpkum 7/FwKL3CUQtyZ7tEWCFyjkaLDjk0bAw4q4VTs= Received: by 10.101.165.35 with SMTP id s35mr5098703ano.258.1284418220642; Mon, 13 Sep 2010 15:50:20 -0700 (PDT) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id q7sm10837809anf.26.2010.09.13.15.50.17 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 13 Sep 2010 15:50:19 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 13 Sep 2010 15:49:25 -0700 From: Pyun YongHyeon Date: Mon, 13 Sep 2010 15:49:25 -0700 To: David Christensen Message-ID: <20100913224925.GK1229@michelle.cdnetworks.com> References: <4C894A76.5040200@tomjudge.com> <20100910002439.GO7203@michelle.cdnetworks.com> <4C8E3D79.6090102@tomjudge.com> <20100913184833.GF1229@michelle.cdnetworks.com> <4C8E775D.8070202@freebsd.org> <20100913193947.GH1229@michelle.cdnetworks.com> <5D267A3F22FD854F8F48B3D2B52381933B540B4C14@IRVEXCHCCR01.corp.ad.broadcom.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5D267A3F22FD854F8F48B3D2B52381933B540B4C14@IRVEXCHCCR01.corp.ad.broadcom.com> User-Agent: Mutt/1.4.2.3i Cc: Tom Judge , "freebsd-net@freebsd.org" , Andre Oppermann , "yongari@freebsd.org" Subject: Re: bce(4) - com_no_buffers (Again) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 22:50:22 -0000 On Mon, Sep 13, 2010 at 03:21:13PM -0700, David Christensen wrote: > > I'm under the impression the header splitting in bce(4) is for > > LRO(opposite of TSO), not for VM magic to enable page flipping > > tricks. > > Header splitting was implemented in the Linux version of bce(4) > to prevent jumbo memory allocations. Allocating 9KB frames was > causing problems on systems used for virtualization. (Harder to > find a contiguous 9KB frame when a hypervisor is in use.) Using > 4KB or smaller buffer sizes was considered more compatible with > virtualization. > > LRO (Large Receive Offload, aka Transparent Packet Aggregation > or TPA on the 10Gb controllers) is not supported on the 1Gb > bce(4) devices. > I meant tcp_lro implementation of FreeBSD. ATM tcp_lro_rx() runs long list of sanity checks before combining TCP segments into a TCP segment but if TCP header is split with its payload I guess we can optimize that path. This way we may be able to support LRO over VLAN, I guess. From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 22:53:44 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A5F02106566B; Mon, 13 Sep 2010 22:53:44 +0000 (UTC) (envelope-from davidch@broadcom.com) Received: from mms2.broadcom.com (mms2.broadcom.com [216.31.210.18]) by mx1.freebsd.org (Postfix) with ESMTP id 7A8818FC15; Mon, 13 Sep 2010 22:53:44 +0000 (UTC) Received: from [10.9.200.131] by mms2.broadcom.com with ESMTP (Broadcom SMTP Relay (Email Firewall v6.3.2)); Mon, 13 Sep 2010 15:21:14 -0700 X-Server-Uuid: D3C04415-6FA8-4F2C-93C1-920E106A2031 Received: from IRVEXCHCCR01.corp.ad.broadcom.com ([10.252.49.30]) by IRVEXCHHUB01.corp.ad.broadcom.com ([10.9.200.131]) with mapi; Mon, 13 Sep 2010 15:21:14 -0700 From: "David Christensen" To: "pyunyh@gmail.com" , "Andre Oppermann" Date: Mon, 13 Sep 2010 15:21:13 -0700 Thread-Topic: bce(4) - com_no_buffers (Again) Thread-Index: ActTe45Bhhr09ZadS66QjMwfeoG7rwAFX1UQ Message-ID: <5D267A3F22FD854F8F48B3D2B52381933B540B4C14@IRVEXCHCCR01.corp.ad.broadcom.com> References: <4C894A76.5040200@tomjudge.com> <20100910002439.GO7203@michelle.cdnetworks.com> <4C8E3D79.6090102@tomjudge.com> <20100913184833.GF1229@michelle.cdnetworks.com> <4C8E775D.8070202@freebsd.org> <20100913193947.GH1229@michelle.cdnetworks.com> In-Reply-To: <20100913193947.GH1229@michelle.cdnetworks.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US MIME-Version: 1.0 X-WSS-ID: 60907C503ES12970608-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Cc: Tom Judge , "freebsd-net@freebsd.org" , "yongari@freebsd.org" Subject: RE: bce(4) - com_no_buffers (Again) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 22:53:44 -0000 > I'm under the impression the header splitting in bce(4) is for > LRO(opposite of TSO), not for VM magic to enable page flipping > tricks. Header splitting was implemented in the Linux version of bce(4) to prevent jumbo memory allocations. Allocating 9KB frames was causing problems on systems used for virtualization. (Harder to find a contiguous 9KB frame when a hypervisor is in use.) Using=20 4KB or smaller buffer sizes was considered more compatible with virtualization. =20 LRO (Large Receive Offload, aka Transparent Packet Aggregation or TPA on the 10Gb controllers) is not supported on the 1Gb=20 bce(4) devices. Dave From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 23:11:34 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9A6DE106564A for ; Mon, 13 Sep 2010 23:11:34 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 658848FC16 for ; Mon, 13 Sep 2010 23:11:34 +0000 (UTC) Received: by pzk7 with SMTP id 7so2689635pzk.13 for ; Mon, 13 Sep 2010 16:11:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=1NURhOgLSituW53Sx58LW8FI2MY2iWK6mQYbOi9wj5A=; b=rXsAc3EVM+xqs5bSixfk2zUCfn8BDxF2SdTyb9JiI/l94rXLT7nmiAprp9LeXNYTQC hGB0K9U+5WchuXgrUAPhEz3/6uPSedP2yA/1PwBS5hPUabITOguU/OgXd/TNST2FVvJ0 vHUtLQ74cxpWzjmkqIn/YiahrO0791cQmo754= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=mXefm22HmSkRUCN8tclzXf/iASoFZHn8i0+vLJuDUCSEi4KZnz4Aqwx9imxRP6kygl +Do2XrQ5psPvuFsjxMYy0z4pKfr0JkWAbPMbyzKA5GSVcCmr6xB92smyBGrIaJjbem48 wVnKr597erTLIkvWRCupy9h+BeAjGpi6cPYKQ= Received: by 10.142.200.2 with SMTP id x2mr361408wff.326.1284419493716; Mon, 13 Sep 2010 16:11:33 -0700 (PDT) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 9sm9325231wfd.0.2010.09.13.16.11.30 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 13 Sep 2010 16:11:31 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 13 Sep 2010 16:10:42 -0700 From: Pyun YongHyeon Date: Mon, 13 Sep 2010 16:10:42 -0700 To: Vlad Galu Message-ID: <20100913231042.GL1229@michelle.cdnetworks.com> References: <20100909102826.GB53812@rambler-co.ru> <20100909201050.GG7203@michelle.cdnetworks.com> <20100909211808.GJ7203@michelle.cdnetworks.com> <20100913142707.GL10050@rambler-co.ru> <20100913180447.GA1229@michelle.cdnetworks.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org, Igor Sysoev Subject: Re: bge hangs on recent 7.3-STABLE X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 23:11:34 -0000 On Tue, Sep 14, 2010 at 01:08:08AM +0300, Vlad Galu wrote: > On Mon, Sep 13, 2010 at 9:04 PM, Pyun YongHyeon wrote: > > On Mon, Sep 13, 2010 at 06:27:08PM +0400, Igor Sysoev wrote: > >> On Thu, Sep 09, 2010 at 02:18:08PM -0700, Pyun YongHyeon wrote: > >> > >> > On Thu, Sep 09, 2010 at 01:10:50PM -0700, Pyun YongHyeon wrote: > >> > > On Thu, Sep 09, 2010 at 02:28:26PM +0400, Igor Sysoev wrote: > >> > > > Hi, > >> > > > > >> > > > I have several hosts running FreeBSD/amd64 7.2-STABLE updated on 11.01.2010 > >> > > > and 25.02.2010. Hosts process about 10K input and 10K output packets/s > >> > > > without issues. One of them, however, is loaded more than others, so it > >> > > > processes 20K/20K packets/s. > >> > > > > >> > > > Recently, I have upgraded one host to 7.3-STABLE, 24.08.2010. > >> > > > Then bge on this host hung two times. I was able to restart it from > >> > > > console using: > >> > > > ? /etc/rc.d/netif restart bge0 > >> > > > > >> > > > Then I have upgraded the most loaded (20K/20K) host to 7.3-STABLE, 07.09.2010. > >> > > > After reboot bge hung every several seconds. I was able to restart it, > >> > > > but bge hung again after several seconds. > >> > > > > >> > > > Then I have downgraded this host to 7.3-STABLE, 14.08.2010, since there > >> > > > were several if_bge.c commits on 15.08.2010. The same hangs. > >> > > > Then I have downgraded this host to 7.3-STABLE, 17.03.2010, before > >> > > > the first if_bge.c commit after 25.02.2010. Now it runs without hangs. > >> > > > > >> > > > The hosts are amd64 dual core SMP with 4G machines. bge information: > >> > > > > >> > > > bge0@pci0:4:0:0: ? ? ? ?class=0x020000 card=0x165914e4 chip=0x165914e4 rev=0x11 hdr=0x00 > >> > > > ? ? vendor ? ? = 'Broadcom Corporation' > >> > > > ? ? device ? ? = 'NetXtreme Gigabit Ethernet PCI Express (BCM5721)' > >> > > > > >> > > > bge0: mem 0xfe5f0000-0xfe5fffff irq 19 at device 0.0 on pci4 > >> > > > miibus1: on bge0 > >> > > > brgphy0: PHY 1 on miibus1 > >> > > > brgphy0: ?10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto > >> > > > bge0: Ethernet address: 00:e0:81:5f:6e:8a > >> > > > > >> > > > >> > > Could you show me verbose boot message(bge part only)? > >> > > Also show me the output of "pciconf -lcbv". > >> > > > >> > > >> > Forgot to send a patch. Let me know whether attached patch fixes > >> > the issue or not. > >> > >> > Index: sys/dev/bge/if_bge.c > >> > =================================================================== > >> > --- sys/dev/bge/if_bge.c ? ?(revision 212341) > >> > +++ sys/dev/bge/if_bge.c ? ?(working copy) > >> > @@ -3386,9 +3386,11 @@ > >> > ? ? sc->bge_rx_saved_considx = rx_cons; > >> > ? ? bge_writembx(sc, BGE_MBX_RX_CONS0_LO, sc->bge_rx_saved_considx); > >> > ? ? if (stdcnt) > >> > - ? ? ? ? ? bge_writembx(sc, BGE_MBX_RX_STD_PROD_LO, sc->bge_std); > >> > + ? ? ? ? ? bge_writembx(sc, BGE_MBX_RX_STD_PROD_LO, (sc->bge_std + > >> > + ? ? ? ? ? ? ? BGE_STD_RX_RING_CNT - 1) % BGE_STD_RX_RING_CNT); > >> > ? ? if (jumbocnt) > >> > - ? ? ? ? ? bge_writembx(sc, BGE_MBX_RX_JUMBO_PROD_LO, sc->bge_jumbo); > >> > + ? ? ? ? ? bge_writembx(sc, BGE_MBX_RX_JUMBO_PROD_LO, (sc->bge_jumbo + > >> > + ? ? ? ? ? ? ? BGE_JUMBO_RX_RING_CNT - 1) % BGE_JUMBO_RX_RING_CNT); > >> > ?#ifdef notyet > >> > ? ? /* > >> > ? ? ?* This register wraps very quickly under heavy packet drops. > >> > >> Thank you, it seems the patch has fixed the bug. > >> BTW, I noticed the same hungs on FreeBSD 8.1, date=2010.09.06.23.59.59 > >> I will apply the patch on all my updated hosts. > >> > > > > Thanks for testing. I'm afraid bge(4) in HEAD, stable/8 and > > stable/7(including 8.1-RELEASE and 7.3-RELEASE) may suffer from > > this issue. Let me know what other hosts work with the patch. > > Hi Pyun, > > Thanks for the patch. It seems to have fixed the symptom in my case, > on a card identical to Igor's, but on board of an IBM eServer 306m. > Thanks for reporting and testing! I really appreciate it. From owner-freebsd-net@FreeBSD.ORG Tue Sep 14 01:15:03 2010 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 71EA5106566B; Tue, 14 Sep 2010 01:15:03 +0000 (UTC) (envelope-from julian@elischer.org) Received: from out-0.mx.aerioconnect.net (out-0-20.mx.aerioconnect.net [216.240.47.80]) by mx1.freebsd.org (Postfix) with ESMTP id 4E6A38FC16; Tue, 14 Sep 2010 01:15:03 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o8E0u66Q000609; Mon, 13 Sep 2010 17:56:06 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id EA9EB2D6017; Mon, 13 Sep 2010 17:56:04 -0700 (PDT) Message-ID: <4C8EC845.2060306@elischer.org> Date: Mon, 13 Sep 2010 17:56:37 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.9) Gecko/20100825 Thunderbird/3.1.3 MIME-Version: 1.0 To: dave@seddon.ca References: <1284107762.5923.306.camel@das8530.vic.bigpond.net.au> <532349FC-9269-4674-872F-FA84292E264C@mimectl> <1284130306.6282.6.camel@das8440.seddon.ca> <009101cb5308$514066d0$f3c13470$@com> <1284423495.5238.99.camel@das8530.vic.bigpond.net.au> In-Reply-To: <1284423495.5238.99.camel@das8530.vic.bigpond.net.au> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: Robert Watson , Andrew Hannam , FreeBSD Net Subject: Re: FreeBSD route tables limited 16? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Sep 2010 01:15:03 -0000 On 9/13/10 5:18 PM, Dave Seddon wrote: > Greetings Julian, > > I've been wondering if it's possible to increase the number of FreeBSD > route tables to a larger number. It seems this is currently 4 bits, > however I was wondering about perhaps 16 bits? Yes the code is designed to handle many more and if you do create more then everything SHOULD handle it. The bottleneck is that we need to store an associated fib with each outgoing (or for that matter incoming) packet, bit we do not at this time want to dedicate a whole word in the mbuf to the task. My "hack" for 8.x (before it was done) was to hide the information in the flags word of the mbuf. I only took 4 bits to make sure I didn't trample on other people's use of bits there. The plan is/was to make a separate entry in the mbuf some time after 7.x branched (say, "now" for example :-) ) you could just steal more bits for now, but if you take 8 bits there will only be one spare. (see /sys/sys/mbuf.h) It may just be time to bite the bullet and steal the entry. Out of curiosity, why do you need > 16 fibs? have you considered using vnet jails a well? > > /* MRT compile-time constants */ > #ifdef _KERNEL > #ifndef ROUTETABLES > #define RT_NUMFIBS 1 > #define RT_MAXFIBS 1 > #else > /* while we use 4 bits in the mbuf flags, we are limited to 16 */ > #define RT_MAXFIBS 16 > #if ROUTETABLES> RT_MAXFIBS > #define RT_NUMFIBS RT_MAXFIBS > #error "ROUTETABLES defined too big" > #else > #if ROUTETABLES == 0 > #define RT_NUMFIBS 1 > #else > #define RT_NUMFIBS ROUTETABLES > #endif > #endif > #endif > #endif > > Really liked your announcement years ago: > http://lists.freebsd.org/pipermail/freebsd-arch/2007-December/007331.html > > Kind regards, > Dave Seddon > +61 447 SEDDON > dave@seddon.ca > > -----Original Message----- > From: Andrew Hannam > To: dave@seddon.ca > Subject: RE: FreeBSD route tables - limited to 16 :( > Date: Mon, 13 Sep 2010 15:55:47 +1000 > Mailer: Microsoft Office Outlook 12.0 > > I think the gentleman is confusing route-tables with routes. > 150K routes is easily possible but it is obvious there is currently only support for up to 16 route tables. > > I think that you are right and the number of bits will need to be updated. > > I don't know the answer to the 'route leaking' question and it has been a long time since I looked at this code. > > You really need to speaking the specialist responsible for the multiple route table code. This person should be clearly marked in the code headers. > > I'm guessing that no-one has thought about using it the way you are planning to use it. > > If I get some time I will have a look - but don't hold your breath. > > Regards, > Andrew. > > -----Original Message----- > From: Dave Seddon [mailto:dave@seddon.ca] > Sent: Saturday, 11 September 2010 12:52 AM > To: Aldous, Matthew D > Cc: dave@seddon.ca; Andrew Hannam; Truman Boyes > Subject: RE: FreeBSD route tables - limited to 16 :( > > Greetings, > > I'm guessing we need to adjust the number of bits defined for the route > table in the mbufs structure definition (where ever that is), then we > can update the route.h to match. > > I guess really we should make the mbufs codes _and_ route.h code pickup > the KERNCONF definition of the variable ROUTETABLES. > > Andrew - thoughts on this? > > I'm not sure if the firewall rules allow you to update the route table > variable in the mbuf, but if it doesn't we should allow this. This > would be equivelant to what they call 'route leaking' in MPLS speak, > when you can pop traffic from one VPN to another (very nasty, but > sometimes handy). yes ipfw does allow you to do this but it needs some more work.. It only really works as the naive user may expect on incoming packets. > > Regards, > Dave > > On Fri, 2010-09-10 at 19:05 +1000, Aldous, Matthew D wrote: >> ________________________________ >> From: Dave Seddon [dave@seddon.ca] >> Sent: Friday, 10 September 2010 6:36 PM >> To: Andrew Hannam >> Cc: dave@seddon.ca; Aldous, Matthew D; Truman Boyes >> Subject: FreeBSD route tables - limited to 16 :( >> >> I just tried compiling up FreeBSD 8.1 with 1024 route tables. It's >> throwing an error, which is tracked down to the >> vi /usr/src/sys/net/route.h (line 99ish). The limit is 16, because as >> the comments say this is 4 bits. Need to look into increasing this to >> say 16 bits :). Given each mbuf will have this, it could cause a >> significant increase in memory usage for a system with a large number of >> packets (although who cares, ram is cheap). >> >> >> /* MRT compile-time constants */ >> #ifdef _KERNEL >> #ifndef ROUTETABLES >> #define RT_NUMFIBS 1 >> #define RT_MAXFIBS 1 >> #else >> /* while we use 4 bits in the mbuf flags, we are limited to 16 */ >> #define RT_MAXFIBS 16 >> #if ROUTETABLES> RT_MAXFIBS >> #define RT_NUMFIBS RT_MAXFIBS >> #error "ROUTETABLES defined too big" >> #else >> #if ROUTETABLES == 0 >> #define RT_NUMFIBS 1 >> #else >> #define RT_NUMFIBS ROUTETABLES >> #endif >> #endif >> #endif >> #endif >> >> > > > From owner-freebsd-net@FreeBSD.ORG Tue Sep 14 07:05:24 2010 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3B383106566B; Tue, 14 Sep 2010 07:05:24 +0000 (UTC) (envelope-from dave@seddon.ca) Received: from postfix.xen.seddon.ca (seddon.ca [203.209.212.18]) by mx1.freebsd.org (Postfix) with ESMTP id 98AA58FC08; Tue, 14 Sep 2010 07:05:22 +0000 (UTC) Received: from [127.0.0.1] (unknown [172.16.100.140]) by postfix.xen.seddon.ca (Postfix) with ESMTP id 17117C11D; Tue, 14 Sep 2010 06:32:11 +0000 (UTC) From: Dave Seddon To: Julian Elischer In-Reply-To: <4C8EC845.2060306@elischer.org> References: <1284107762.5923.306.camel@das8530.vic.bigpond.net.au> <532349FC-9269-4674-872F-FA84292E264C@mimectl> <1284130306.6282.6.camel@das8440.seddon.ca> <009101cb5308$514066d0$f3c13470$@com> <1284423495.5238.99.camel@das8530.vic.bigpond.net.au> <4C8EC845.2060306@elischer.org> Content-Type: text/plain; charset="UTF-8" Date: Tue, 14 Sep 2010 16:31:39 +1000 Message-ID: <1284445899.5238.155.camel@das8530.vic.bigpond.net.au> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , Robert Watson , Andrew Hannam , dave@seddon.ca Subject: Re: FreeBSD route tables limited 16? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: dave@seddon.ca List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Sep 2010 07:05:24 -0000 Greetings, Thanks for the quick response. It sounds like dedicating some space for this in the mbuf would be the best way forward, but the question is how much. I'm worried that most freebsd users won't go for lots of route tables, which is why you went for 4 bits originally. Within the network service provider space there is frequently a requirement for lots of virtual-routing with MPLS. I imagine there are others in my situation, including vendors and people working on equipment like Cisco/Juniper/Lucatel. Regarding the size to dedicate, the best number might be 12 bits or 4096. This would allow a route table per VLAN on a 802.1q interface. (Actually I'm lying a little because the first and last vlan IDs aren't usable :) ). Perhaps a separate option for non-common users who want many route tables would be best. e.g. GIANT_ROUTETABLES=12 Seems like there would need to be changes in multiple places although perhaps this list isn't exhaustive. So far the files to edit are: /usr/src/sys/net/route.h /sys/sys/mbuf.h Regarding firewalls and these multiple route tables, have you considered having a separate firewall rule table per route table? I haven't looked at the vnet jails, yet. Will do. Thanks. Kind regards, Dave -----Original Message----- From: Julian Elischer To: dave@seddon.ca Cc: Andrew Hannam , FreeBSD Net , Robert Watson Subject: Re: FreeBSD route tables limited 16? Date: Mon, 13 Sep 2010 17:56:37 -0700 Mailer: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.9) Gecko/20100825 Thunderbird/3.1.3 On 9/13/10 5:18 PM, Dave Seddon wrote: > Greetings Julian, > > I've been wondering if it's possible to increase the number of FreeBSD > route tables to a larger number. It seems this is currently 4 bits, > however I was wondering about perhaps 16 bits? Yes the code is designed to handle many more and if you do create more then everything SHOULD handle it. The bottleneck is that we need to store an associated fib with each outgoing (or for that matter incoming) packet, bit we do not at this time want to dedicate a whole word in the mbuf to the task. My "hack" for 8.x (before it was done) was to hide the information in the flags word of the mbuf. I only took 4 bits to make sure I didn't trample on other people's use of bits there. The plan is/was to make a separate entry in the mbuf some time after 7.x branched (say, "now" for example :-) ) you could just steal more bits for now, but if you take 8 bits there will only be one spare. (see /sys/sys/mbuf.h) It may just be time to bite the bullet and steal the entry. Out of curiosity, why do you need > 16 fibs? have you considered using vnet jails a well? > > /* MRT compile-time constants */ > #ifdef _KERNEL > #ifndef ROUTETABLES > #define RT_NUMFIBS 1 > #define RT_MAXFIBS 1 > #else > /* while we use 4 bits in the mbuf flags, we are limited to 16 */ > #define RT_MAXFIBS 16 > #if ROUTETABLES> RT_MAXFIBS > #define RT_NUMFIBS RT_MAXFIBS > #error "ROUTETABLES defined too big" > #else > #if ROUTETABLES == 0 > #define RT_NUMFIBS 1 > #else > #define RT_NUMFIBS ROUTETABLES > #endif > #endif > #endif > #endif > > Really liked your announcement years ago: > http://lists.freebsd.org/pipermail/freebsd-arch/2007-December/007331.html > > Kind regards, > Dave Seddon > +61 447 SEDDON > dave@seddon.ca > > -----Original Message----- > From: Andrew Hannam > To: dave@seddon.ca > Subject: RE: FreeBSD route tables - limited to 16 :( > Date: Mon, 13 Sep 2010 15:55:47 +1000 > Mailer: Microsoft Office Outlook 12.0 > > I think the gentleman is confusing route-tables with routes. > 150K routes is easily possible but it is obvious there is currently only support for up to 16 route tables. > > I think that you are right and the number of bits will need to be updated. > > I don't know the answer to the 'route leaking' question and it has been a long time since I looked at this code. > > You really need to speaking the specialist responsible for the multiple route table code. This person should be clearly marked in the code headers. > > I'm guessing that no-one has thought about using it the way you are planning to use it. > > If I get some time I will have a look - but don't hold your breath. > > Regards, > Andrew. > > -----Original Message----- > From: Dave Seddon [mailto:dave@seddon.ca] > Sent: Saturday, 11 September 2010 12:52 AM > To: Aldous, Matthew D > Cc: dave@seddon.ca; Andrew Hannam; Truman Boyes > Subject: RE: FreeBSD route tables - limited to 16 :( > > Greetings, > > I'm guessing we need to adjust the number of bits defined for the route > table in the mbufs structure definition (where ever that is), then we > can update the route.h to match. > > I guess really we should make the mbufs codes _and_ route.h code pickup > the KERNCONF definition of the variable ROUTETABLES. > > Andrew - thoughts on this? > > I'm not sure if the firewall rules allow you to update the route table > variable in the mbuf, but if it doesn't we should allow this. This > would be equivelant to what they call 'route leaking' in MPLS speak, > when you can pop traffic from one VPN to another (very nasty, but > sometimes handy). yes ipfw does allow you to do this but it needs some more work.. It only really works as the naive user may expect on incoming packets. > > Regards, > Dave > > On Fri, 2010-09-10 at 19:05 +1000, Aldous, Matthew D wrote: >> ________________________________ >> From: Dave Seddon [dave@seddon.ca] >> Sent: Friday, 10 September 2010 6:36 PM >> To: Andrew Hannam >> Cc: dave@seddon.ca; Aldous, Matthew D; Truman Boyes >> Subject: FreeBSD route tables - limited to 16 :( >> >> I just tried compiling up FreeBSD 8.1 with 1024 route tables. It's >> throwing an error, which is tracked down to the >> vi /usr/src/sys/net/route.h (line 99ish). The limit is 16, because as >> the comments say this is 4 bits. Need to look into increasing this to >> say 16 bits :). Given each mbuf will have this, it could cause a >> significant increase in memory usage for a system with a large number of >> packets (although who cares, ram is cheap). >> >> >> /* MRT compile-time constants */ >> #ifdef _KERNEL >> #ifndef ROUTETABLES >> #define RT_NUMFIBS 1 >> #define RT_MAXFIBS 1 >> #else >> /* while we use 4 bits in the mbuf flags, we are limited to 16 */ >> #define RT_MAXFIBS 16 >> #if ROUTETABLES> RT_MAXFIBS >> #define RT_NUMFIBS RT_MAXFIBS >> #error "ROUTETABLES defined too big" >> #else >> #if ROUTETABLES == 0 >> #define RT_NUMFIBS 1 >> #else >> #define RT_NUMFIBS ROUTETABLES >> #endif >> #endif >> #endif >> #endif >> >> > > > From owner-freebsd-net@FreeBSD.ORG Tue Sep 14 09:33:23 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 53AA6106566C; Tue, 14 Sep 2010 09:33:23 +0000 (UTC) (envelope-from fabien.thomas@netasq.com) Received: from work.netasq.com (mars.netasq.com [91.212.116.3]) by mx1.freebsd.org (Postfix) with ESMTP id 72DA38FC12; Tue, 14 Sep 2010 09:33:21 +0000 (UTC) Received: from [10.20.1.1] (unknown [10.2.27.254]) by work.netasq.com (Postfix) with ESMTPSA id 98AA2740015; Tue, 14 Sep 2010 11:17:47 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v1081) Content-Type: text/plain; charset=us-ascii From: Fabien Thomas In-Reply-To: <4C8E0C1E.2020707@networx.ch> Date: Tue, 14 Sep 2010 11:18:14 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4C8E0C1E.2020707@networx.ch> To: Andre Oppermann X-Mailer: Apple Mail (2.1081) Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org Subject: Re: TCP loopback socket fusing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Sep 2010 09:33:23 -0000 Great, This will maybe kill the long time debate about "my loopback is slow vs = linux" To have the best of both world what about a socket option to = enable/disable fusing: can be useful when you need to see some connection "packetized". Fabien On 13 sept. 2010, at 13:33, Andre Oppermann wrote: > When a TCP connection via loopback back to localhost is made the whole > send, segmentation and receive path (with larger packets though) is = still > executed. This has some considerable overhead. >=20 > To short-circuit the send and receive sockets on localhost TCP = connections > I've made a proof-of-concept patch that directly places the data in = the > other side's socket buffer without doing any packetization and other = protocol > overhead (like UNIX domain sockets). The connections setup (SYN, = SYN-ACK, > ACK) and shutdown are still handled by normal TCP segments via = loopback so > that firewalling stills works. The actual payload data during the = session > won't be seen and the sequence numbers don't move other than for SYN = and FIN. > The sequence are remain valid though. Obviously tcpdump won't see any = data > transfers either if the connection has fused sockets. >=20 > Preliminary testing (with WITNESS and INVARIANTS enabled) has shown = stable > operation and a rough doubling of the throughput on loopback = connections. > I've tested most socket teardown cases and it behaves fine. I'm not = entirely > sure I've got all possible path's but the way it is integrated should = properly > defuse the sockets in all situations. >=20 > Testers and feedback wanted: >=20 > http://people.freebsd.org/~andre/tcp_loopfuse-20100913.diff >=20 > --=20 > Andre >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Sep 14 09:35:54 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B75A61065679; Tue, 14 Sep 2010 09:35:54 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 763A08FC14; Tue, 14 Sep 2010 09:35:54 +0000 (UTC) Received: by iwn34 with SMTP id 34so6779221iwn.13 for ; Tue, 14 Sep 2010 02:35:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:cc:content-type; bh=u0kR7tyWZTN0WlcOVB4A2262sYK3Tr6H4+Rowu48GaU=; b=IkxnNIWta7zfg2Exf1AC3xPRjqT8pAzAvvnJzznF/WRlBqmliAsIgA7OAb8vXRIy4J zt2jVeKmu1pxux7lA7tHrgSA//i4VUMHQAUyLJzhpPJ6KSsoKzsDK9eJitkOn+RzXgU5 RkDW88p8vzoPvKwf3mWddsQ9Bt1cMY5P/euhc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:cc:content-type; b=NDD2pdcnr5irtr479Fv3/HcI7pHRnT/aLlLjpMeathBSEaeWpNRMnd/ZON6JT1gtof UCtME6hiisXS7iLerzq/C6788FicPR7Oh/EB1MPvzKzA2198N35EYLo7wGIIKRyXokfK DU59BNfWiQg07GUYY39TRB4ixTzouF0z2rR+w= MIME-Version: 1.0 Received: by 10.231.190.9 with SMTP id dg9mr7721593ibb.54.1284456953453; Tue, 14 Sep 2010 02:35:53 -0700 (PDT) Received: by 10.231.156.206 with HTTP; Tue, 14 Sep 2010 02:35:53 -0700 (PDT) Date: Tue, 14 Sep 2010 17:35:53 +0800 Message-ID: From: Adrian Chadd To: freebsd-current Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net Subject: AR9132 CPU and AR9100 wireless support X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Sep 2010 09:35:54 -0000 Hi everyone, I've just pushed the initial support for the AR9100 wireless MAC into my git repository. This is for the WMAC on the AR9132 SoC. I've tested it in 11bg hostap mode on an AP83 derived box - the TP-Link TL-WR1043ND. The source tree has support for the CPU, ethernet (but not the switch PHY; it's enough to get data across it!); flash and the AR9100 WMAC. I've only tested "open" hostap mode on 11bg on a fixed channel. The GIT repo is at: http://www.gitorious.org/~adrianchadd/freebsd/adrianchadd-freebsd ; it's the "work/atheros" branch. You'll need to open the unit up, solder on some pins to get to the serial port and acquire a TTL -> RS232 level converter. There's pictures and howto on the OpenWRT wiki: http://wiki.openwrt.org/toh/tp-link/tl-wr1043nd Ciao! Adrian From owner-freebsd-net@FreeBSD.ORG Tue Sep 14 09:50:11 2010 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 55C0E1065675; Tue, 14 Sep 2010 09:50:11 +0000 (UTC) (envelope-from "."@babolo.ru) Received: from smtp1.babolo.ru (smtp1.babolo.ru [195.9.14.139]) by mx1.freebsd.org (Postfix) with ESMTP id CFDF18FC0C; Tue, 14 Sep 2010 09:50:10 +0000 (UTC) Received: from cicuta.babolo.ru ([194.58.246.5]) by smtp1.babolo.ru (8.14.2/8.14.2) with SMTP id o8E9TYG3088640; Tue, 14 Sep 2010 13:29:34 +0400 (MSD) (envelope-from .@babolo.ru) Received: (nullmailer pid 35047 invoked by uid 136); Tue, 14 Sep 2010 09:27:32 -0000 Date: Tue, 14 Sep 2010 13:27:32 +0400 From: Aleksandr A Babaylov <.@babolo.ru> To: Dave Seddon Message-ID: <20100914092732.GA34995@babolo.ru> References: <1284107762.5923.306.camel@das8530.vic.bigpond.net.au> <532349FC-9269-4674-872F-FA84292E264C@mimectl> <1284130306.6282.6.camel@das8440.seddon.ca> <009101cb5308$514066d0$f3c13470$@com> <1284423495.5238.99.camel@das8530.vic.bigpond.net.au> <4C8EC845.2060306@elischer.org> <1284445899.5238.155.camel@das8530.vic.bigpond.net.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1284445899.5238.155.camel@das8530.vic.bigpond.net.au> Cc: Andrew Hannam , Robert Watson , Julian Elischer , FreeBSD Net Subject: Re: FreeBSD route tables limited 16? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Sep 2010 09:50:11 -0000 On Tue, Sep 14, 2010 at 04:31:39PM +1000, Dave Seddon wrote: > It sounds like dedicating some space for this in the mbuf would be the > best way forward, but the question is how much. I'm worried that most > freebsd users won't go for lots of route tables, which is why you went > for 4 bits originally. > > Within the network service provider space there is frequently a > requirement for lots of virtual-routing with MPLS. I imagine there are > others in my situation, including vendors and people working on > equipment like Cisco/Juniper/Lucatel. > > Regarding the size to dedicate, the best number might be 12 bits or > 4096. This would allow a route table per VLAN on a 802.1q interface. > (Actually I'm lying a little because the first and last vlan IDs aren't > usable :) ). > > Perhaps a separate option for non-common users who want many route > tables would be best. e.g. > > GIANT_ROUTETABLES=12 Why not 16? There can be several independent iface with VLAN on each. arg1 in ipfw has 16 bits for fib number. Yes, it is very special case when huge number of fib needed. From owner-freebsd-net@FreeBSD.ORG Tue Sep 14 10:35:52 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 39034106566B for ; Tue, 14 Sep 2010 10:35:52 +0000 (UTC) (envelope-from mdounin@mdounin.ru) Received: from mdounin.cust.ramtel.ru (mdounin.cust.ramtel.ru [81.19.69.81]) by mx1.freebsd.org (Postfix) with ESMTP id DACAD8FC15 for ; Tue, 14 Sep 2010 10:35:51 +0000 (UTC) Received: from mdounin.ru (mdounin.cust.ramtel.ru [81.19.69.81]) by mdounin.cust.ramtel.ru (Postfix) with ESMTP id C519217015; Tue, 14 Sep 2010 14:35:49 +0400 (MSD) Date: Tue, 14 Sep 2010 14:35:49 +0400 From: Maxim Dounin To: Ian FREISLICH Message-ID: <20100914103549.GI99657@mdounin.ru> References: <4C8E0C1E.2020707@networx.ch> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org, Fabien Thomas Subject: Re: TCP loopback socket fusing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Sep 2010 10:35:52 -0000 Hello! On Tue, Sep 14, 2010 at 12:12:03PM +0200, Ian FREISLICH wrote: > Fabien Thomas wrote: > > Great, > > > > This will maybe kill the long time debate about "my loopback is slow vs > > linux" > > To have the best of both world what about a socket option to > > enable/disable fusing: > > can be useful when you need to see some connection "packetized". > > To chime in, I had a "slow" loopback issue earlier this week. It > turned out the problem was caused by delayed ack on the loopback > where the client didn't need to transmit any data to the server. > It delayed each packet from the server by 100ms. After patching > the server to: > > setsockopt(desc->accept_fd, IPPROTO_TCP, TCP_NODELAY, &x, sizeof(x)); > > It's now faster than on linux. > > Perhaps this is one of the causes of "my loopback is slow vs linux". > > FWIW, I couldn't find a way to turn off dealyed_ack on just loopback > interface. AFAIK in linux delayed ack behaves a bit more gently and doesn't delay first ack(s) in a connection. As a result linux hides some classic delayed ack vs. Nagle problems. Something similar probably should be adapted. Maxim Dounin From owner-freebsd-net@FreeBSD.ORG Tue Sep 14 10:52:40 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E1E23106566C for ; Tue, 14 Sep 2010 10:52:40 +0000 (UTC) (envelope-from ianf@clue.co.za) Received: from inbound01.jnb1.gp-online.net (inbound01.jnb1.gp-online.net [41.161.16.135]) by mx1.freebsd.org (Postfix) with ESMTP id 784D28FC1B for ; Tue, 14 Sep 2010 10:52:40 +0000 (UTC) Received: from [41.154.88.19] (helo=clue.co.za) by inbound01.jnb1.gp-online.net with esmtpsa (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.63) (envelope-from ) id 1OvSUj-0005aB-KJ; Tue, 14 Sep 2010 12:12:10 +0200 Received: from localhost ([127.0.0.1] helo=clue.co.za) by clue.co.za with esmtp (Exim 4.72 (FreeBSD)) (envelope-from ) id 1OvSUd-0000mU-0l; Tue, 14 Sep 2010 12:12:03 +0200 Message-Id: To: Fabien Thomas From: Ian FREISLICH In-Reply-To: References: <4C8E0C1E.2020707@networx.ch> X-Attribution: BOFH Date: Tue, 14 Sep 2010 12:12:03 +0200 Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org Subject: Re: TCP loopback socket fusing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Sep 2010 10:52:41 -0000 Fabien Thomas wrote: > Great, > > This will maybe kill the long time debate about "my loopback is slow vs > linux" > To have the best of both world what about a socket option to > enable/disable fusing: > can be useful when you need to see some connection "packetized". To chime in, I had a "slow" loopback issue earlier this week. It turned out the problem was caused by delayed ack on the loopback where the client didn't need to transmit any data to the server. It delayed each packet from the server by 100ms. After patching the server to: setsockopt(desc->accept_fd, IPPROTO_TCP, TCP_NODELAY, &x, sizeof(x)); It's now faster than on linux. Perhaps this is one of the causes of "my loopback is slow vs linux". FWIW, I couldn't find a way to turn off dealyed_ack on just loopback interface. Ian -- Ian Freislich From owner-freebsd-net@FreeBSD.ORG Tue Sep 14 15:41:03 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 387011065672 for ; Tue, 14 Sep 2010 15:41:03 +0000 (UTC) (envelope-from oppermann@networx.ch) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 86C8F8FC15 for ; Tue, 14 Sep 2010 15:41:02 +0000 (UTC) Received: (qmail 62465 invoked from network); 14 Sep 2010 15:36:00 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 14 Sep 2010 15:36:00 -0000 Message-ID: <4C8F978F.1030707@networx.ch> Date: Tue, 14 Sep 2010 17:41:03 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: Fabien Thomas References: <4C8E0C1E.2020707@networx.ch> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org Subject: Re: TCP loopback socket fusing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Sep 2010 15:41:03 -0000 On 14.09.2010 11:18, Fabien Thomas wrote: > Great, > > This will maybe kill the long time debate about "my loopback is slow vs linux" > To have the best of both world what about a socket option to enable/disable fusing: > can be useful when you need to see some connection "packetized". A sysctl to that effect is already in the patch. -- Andre > Fabien > > On 13 sept. 2010, at 13:33, Andre Oppermann wrote: > >> When a TCP connection via loopback back to localhost is made the whole >> send, segmentation and receive path (with larger packets though) is still >> executed. This has some considerable overhead. >> >> To short-circuit the send and receive sockets on localhost TCP connections >> I've made a proof-of-concept patch that directly places the data in the >> other side's socket buffer without doing any packetization and other protocol >> overhead (like UNIX domain sockets). The connections setup (SYN, SYN-ACK, >> ACK) and shutdown are still handled by normal TCP segments via loopback so >> that firewalling stills works. The actual payload data during the session >> won't be seen and the sequence numbers don't move other than for SYN and FIN. >> The sequence are remain valid though. Obviously tcpdump won't see any data >> transfers either if the connection has fused sockets. >> >> Preliminary testing (with WITNESS and INVARIANTS enabled) has shown stable >> operation and a rough doubling of the throughput on loopback connections. >> I've tested most socket teardown cases and it behaves fine. I'm not entirely >> sure I've got all possible path's but the way it is integrated should properly >> defuse the sockets in all situations. >> >> Testers and feedback wanted: >> >> http://people.freebsd.org/~andre/tcp_loopfuse-20100913.diff >> >> -- >> Andre >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > From owner-freebsd-net@FreeBSD.ORG Tue Sep 14 16:05:12 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7087C1065670 for ; Tue, 14 Sep 2010 16:05:12 +0000 (UTC) (envelope-from oppermann@networx.ch) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id D2AE58FC14 for ; Tue, 14 Sep 2010 16:05:11 +0000 (UTC) Received: (qmail 62703 invoked from network); 14 Sep 2010 16:00:09 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 14 Sep 2010 16:00:09 -0000 Message-ID: <4C8F9D38.9020100@networx.ch> Date: Tue, 14 Sep 2010 18:05:12 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: Ian FREISLICH References: <4C8E0C1E.2020707@networx.ch> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org, Fabien Thomas Subject: Re: TCP loopback socket fusing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Sep 2010 16:05:12 -0000 On 14.09.2010 12:12, Ian FREISLICH wrote: > Fabien Thomas wrote: >> Great, >> >> This will maybe kill the long time debate about "my loopback is slow vs >> linux" >> To have the best of both world what about a socket option to >> enable/disable fusing: >> can be useful when you need to see some connection "packetized". > > To chime in, I had a "slow" loopback issue earlier this week. It > turned out the problem was caused by delayed ack on the loopback > where the client didn't need to transmit any data to the server. > It delayed each packet from the server by 100ms. After patching > the server to: > > setsockopt(desc->accept_fd, IPPROTO_TCP, TCP_NODELAY,&x, sizeof(x)); > > It's now faster than on linux. > > Perhaps this is one of the causes of "my loopback is slow vs linux". > > FWIW, I couldn't find a way to turn off dealyed_ack on just loopback > interface. Good point. You can't at the moment but it certainly makes a lot of sense. Let me see what I can come up with. -- Andre From owner-freebsd-net@FreeBSD.ORG Tue Sep 14 16:07:18 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4D2BE106564A for ; Tue, 14 Sep 2010 16:07:18 +0000 (UTC) (envelope-from oppermann@networx.ch) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id B14398FC08 for ; Tue, 14 Sep 2010 16:07:17 +0000 (UTC) Received: (qmail 62745 invoked from network); 14 Sep 2010 16:02:15 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 14 Sep 2010 16:02:15 -0000 Message-ID: <4C8F9DB6.4050309@networx.ch> Date: Tue, 14 Sep 2010 18:07:18 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: Maxim Dounin References: <4C8E0C1E.2020707@networx.ch> <20100914103549.GI99657@mdounin.ru> In-Reply-To: <20100914103549.GI99657@mdounin.ru> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Ian FREISLICH , Fabien Thomas , freebsd-current@freebsd.org Subject: Re: TCP loopback socket fusing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Sep 2010 16:07:18 -0000 On 14.09.2010 12:35, Maxim Dounin wrote: > Hello! > > On Tue, Sep 14, 2010 at 12:12:03PM +0200, Ian FREISLICH wrote: > >> Fabien Thomas wrote: >>> Great, >>> >>> This will maybe kill the long time debate about "my loopback is slow vs >>> linux" >>> To have the best of both world what about a socket option to >>> enable/disable fusing: >>> can be useful when you need to see some connection "packetized". >> >> To chime in, I had a "slow" loopback issue earlier this week. It >> turned out the problem was caused by delayed ack on the loopback >> where the client didn't need to transmit any data to the server. >> It delayed each packet from the server by 100ms. After patching >> the server to: >> >> setsockopt(desc->accept_fd, IPPROTO_TCP, TCP_NODELAY,&x, sizeof(x)); >> >> It's now faster than on linux. >> >> Perhaps this is one of the causes of "my loopback is slow vs linux". >> >> FWIW, I couldn't find a way to turn off dealyed_ack on just loopback >> interface. > > AFAIK in linux delayed ack behaves a bit more gently and doesn't > delay first ack(s) in a connection. As a result linux hides some > classic delayed ack vs. Nagle problems. > > Something similar probably should be adapted. I saw something like that while glancing over the Linux code some time ago. Couldn't make much sense out of the code snipped because their TCP code split into a myriad of small functions and thus hard to follow in the beginning. Not the ours is much easier on a beginner. -- Andre From owner-freebsd-net@FreeBSD.ORG Tue Sep 14 16:08:37 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1140E1065673; Tue, 14 Sep 2010 16:08:37 +0000 (UTC) (envelope-from fabien.thomas@netasq.com) Received: from work.netasq.com (mars.netasq.com [91.212.116.3]) by mx1.freebsd.org (Postfix) with ESMTP id 474688FC22; Tue, 14 Sep 2010 16:08:35 +0000 (UTC) Received: from [10.20.1.1] (unknown [10.2.27.254]) by work.netasq.com (Postfix) with ESMTPSA id 32EFF740002; Tue, 14 Sep 2010 18:08:06 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v1081) Content-Type: text/plain; charset=us-ascii From: Fabien Thomas In-Reply-To: <4C8F978F.1030707@networx.ch> Date: Tue, 14 Sep 2010 18:08:34 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <89E74A8F-4FF9-482B-83D0-3D076F0E41E4@netasq.com> References: <4C8E0C1E.2020707@networx.ch> <4C8F978F.1030707@networx.ch> To: Andre Oppermann X-Mailer: Apple Mail (2.1081) Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org Subject: Re: TCP loopback socket fusing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Sep 2010 16:08:37 -0000 On 14 sept. 2010, at 17:41, Andre Oppermann wrote: > On 14.09.2010 11:18, Fabien Thomas wrote: >> Great, >>=20 >> This will maybe kill the long time debate about "my loopback is slow = vs linux" >> To have the best of both world what about a socket option to = enable/disable fusing: >> can be useful when you need to see some connection "packetized". >=20 > A sysctl to that effect is already in the patch. yes, i'm just wondering on a per connection setting. >=20 > --=20 > Andre >=20 >> Fabien >>=20 >> On 13 sept. 2010, at 13:33, Andre Oppermann wrote: >>=20 >>> When a TCP connection via loopback back to localhost is made the = whole >>> send, segmentation and receive path (with larger packets though) is = still >>> executed. This has some considerable overhead. >>>=20 >>> To short-circuit the send and receive sockets on localhost TCP = connections >>> I've made a proof-of-concept patch that directly places the data in = the >>> other side's socket buffer without doing any packetization and other = protocol >>> overhead (like UNIX domain sockets). The connections setup (SYN, = SYN-ACK, >>> ACK) and shutdown are still handled by normal TCP segments via = loopback so >>> that firewalling stills works. The actual payload data during the = session >>> won't be seen and the sequence numbers don't move other than for SYN = and FIN. >>> The sequence are remain valid though. Obviously tcpdump won't see = any data >>> transfers either if the connection has fused sockets. >>>=20 >>> Preliminary testing (with WITNESS and INVARIANTS enabled) has shown = stable >>> operation and a rough doubling of the throughput on loopback = connections. >>> I've tested most socket teardown cases and it behaves fine. I'm not = entirely >>> sure I've got all possible path's but the way it is integrated = should properly >>> defuse the sockets in all situations. >>>=20 >>> Testers and feedback wanted: >>>=20 >>> http://people.freebsd.org/~andre/tcp_loopfuse-20100913.diff >>>=20 >>> -- >>> Andre >>>=20 >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>=20 >>=20 >>=20 >=20 From owner-freebsd-net@FreeBSD.ORG Tue Sep 14 16:22:45 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85551106564A for ; Tue, 14 Sep 2010 16:22:45 +0000 (UTC) (envelope-from oppermann@networx.ch) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id E696B8FC17 for ; Tue, 14 Sep 2010 16:22:44 +0000 (UTC) Received: (qmail 62857 invoked from network); 14 Sep 2010 16:17:42 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 14 Sep 2010 16:17:42 -0000 Message-ID: <4C8FA155.8050602@networx.ch> Date: Tue, 14 Sep 2010 18:22:45 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: Fabien Thomas References: <4C8E0C1E.2020707@networx.ch> <4C8F978F.1030707@networx.ch> <89E74A8F-4FF9-482B-83D0-3D076F0E41E4@netasq.com> In-Reply-To: <89E74A8F-4FF9-482B-83D0-3D076F0E41E4@netasq.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org Subject: Re: TCP loopback socket fusing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Sep 2010 16:22:45 -0000 On 14.09.2010 18:08, Fabien Thomas wrote: > > On 14 sept. 2010, at 17:41, Andre Oppermann wrote: > >> On 14.09.2010 11:18, Fabien Thomas wrote: >>> Great, >>> >>> This will maybe kill the long time debate about "my loopback is slow vs linux" >>> To have the best of both world what about a socket option to enable/disable fusing: >>> can be useful when you need to see some connection "packetized". >> >> A sysctl to that effect is already in the patch. > yes, i'm just wondering on a per connection setting. I've devised a way to prevent socket fusing when bpf is enabled on the interface the loopback came from. So I'm leaning against adding another obscure and non-portable socket option. -- Andre From owner-freebsd-net@FreeBSD.ORG Tue Sep 14 16:32:19 2010 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 46AA8106564A; Tue, 14 Sep 2010 16:32:19 +0000 (UTC) (envelope-from julian@elischer.org) Received: from out-0.mx.aerioconnect.net (out-0-35.mx.aerioconnect.net [216.240.47.95]) by mx1.freebsd.org (Postfix) with ESMTP id 222188FC0C; Tue, 14 Sep 2010 16:32:18 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o8EGWHRM027204; Tue, 14 Sep 2010 09:32:17 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id E525E2D601A; Tue, 14 Sep 2010 09:32:16 -0700 (PDT) Message-ID: <4C8FA3B2.4070204@elischer.org> Date: Tue, 14 Sep 2010 09:32:50 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.9) Gecko/20100825 Thunderbird/3.1.3 MIME-Version: 1.0 To: dave@seddon.ca References: <1284107762.5923.306.camel@das8530.vic.bigpond.net.au> <532349FC-9269-4674-872F-FA84292E264C@mimectl> <1284130306.6282.6.camel@das8440.seddon.ca> <009101cb5308$514066d0$f3c13470$@com> <1284423495.5238.99.camel@das8530.vic.bigpond.net.au> <4C8EC845.2060306@elischer.org> <1284445899.5238.155.camel@das8530.vic.bigpond.net.au> In-Reply-To: <1284445899.5238.155.camel@das8530.vic.bigpond.net.au> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: Robert Watson , Andrew Hannam , FreeBSD Net Subject: Re: FreeBSD route tables limited 16? (mbuf changes) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Sep 2010 16:32:19 -0000 On 9/13/10 11:31 PM, Dave Seddon wrote: > Greetings, > > Thanks for the quick response. > > It sounds like dedicating some space for this in the mbuf would be the > best way forward, but the question is how much. I'm worried that most > freebsd users won't go for lots of route tables, which is why you went > for 4 bits originally. > > Within the network service provider space there is frequently a > requirement for lots of virtual-routing with MPLS. I imagine there are > others in my situation, including vendors and people working on > equipment like Cisco/Juniper/Lucatel. > > Regarding the size to dedicate, the best number might be 12 bits or > 4096. This would allow a route table per VLAN on a 802.1q interface. > (Actually I'm lying a little because the first and last vlan IDs aren't > usable :) ). > > Perhaps a separate option for non-common users who want many route > tables would be best. e.g. > > GIANT_ROUTETABLES=12 possibly but see below. note: it's not giant tables, but MANY tables. options LOTSA_FIBS # Use more than 4 bits to enumerate mbuf fibs. > > Seems like there would need to be changes in multiple places although > perhaps this list isn't exhaustive. So far the files to edit are: > /usr/src/sys/net/route.h > /sys/sys/mbuf.h Basically the definition of how fibs are stored in an mbuf is encapsulated in mbuf.h and a macro is used to do it everywhere. so the main damage is limited to that file. You shouldn't really do it with an option because modules built with one option would crash when mixed with a kernel built with the other option, which is problematic for support. We only allow modules and kernel to be incompatible across a version change. There was talk about revamping mbufs for 9.x. I think Robert, Andre , Randall and some of the others may want to comment on that. > > > Regarding firewalls and these multiple route tables, have you considered > having a separate firewall rule table per route table? no, but there is a separate ipfw rule-set per vnet-jail. eventually if you keep adding more and more feature per-routing table, you end up recreating the vnet-jail feature. > > > I haven't looked at the vnet jails, yet. Will do. Thanks. > > Kind regards, > Dave > From owner-freebsd-net@FreeBSD.ORG Wed Sep 15 03:16:46 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 859C6106564A; Wed, 15 Sep 2010 03:16:46 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3D9028FC19; Wed, 15 Sep 2010 03:16:45 +0000 (UTC) Received: by iwn34 with SMTP id 34so7631710iwn.13 for ; Tue, 14 Sep 2010 20:16:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=8EqNgsU7LJ1eqyorXOq+Teutm2ORY1IsUfJZ2U1WB3s=; b=Ms0YRUOL1mykZcIzd3rE26bJNbBBW62qgFHdsFMWNPhvsB/+xAO/zRjrWJnft+0dy+ a7Yf6Dv3P4nfBzvyMhQLXvZbw2F1q2AkOhr0MXpP7vsmP5/O7bPT285PeVI32PqrRaUP WKIOv4JTkcXS/0asF5fqL6lpT7hplxw/P1BK8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=qheKhaOAy5ZsIDEH/sW7VWxWZy1Syv+FowLFGMn5qGlQAthQq0N2gHQbzAuRr4sUVb PaHe23WGNTHxeC2GjhO1BommcID5WPoB1BS7cod6uqbqFMklvfNX1N1HA+fCGygZ2OTS BMUsGktn4aT97cViLREOkONAvw7NpeuilE21g= MIME-Version: 1.0 Received: by 10.231.15.73 with SMTP id j9mr1002404iba.23.1284520605403; Tue, 14 Sep 2010 20:16:45 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.231.156.206 with HTTP; Tue, 14 Sep 2010 20:16:45 -0700 (PDT) In-Reply-To: <9EB9CCE4-2F4A-4653-BF11-066E3490F3AF@FreeBSD.org> References: <1F563B4F-B1DF-4829-966D-63264EC8AB61@jnielsen.net> <9EB9CCE4-2F4A-4653-BF11-066E3490F3AF@FreeBSD.org> Date: Wed, 15 Sep 2010 11:16:45 +0800 X-Google-Sender-Auth: uE5lMS9PQfaNOeT_ClXqIa9EZP4 Message-ID: From: Adrian Chadd To: Rui Paulo Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org, John Nielsen Subject: Re: Is 802.11n rate control being worked on? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Sep 2010 03:16:46 -0000 Hi John, I'm working on bringing over the changes from Linux ath9k into our HAL. I'm slowly starting on bringing over simple bits and pieces but I hope to eventually be able to bring over large chunks of the hardware fiddling almost untouched. Since the current open Atheros development by people with the docs is occuring in linux ath9k, being able to sync against that is high up on my todo list. Rui told me his main problem was lacking in reliable driver code to actually make/receive 11n frames reliably. I'm hoping to just lift the ath9k hardware code but that doesn't help with the needed net80211 changes to support 11n. If you're happy to take over rui's 11n work, I'm happy working on porting over ath9k driver/rate changes. Adrian On 8 September 2010 20:32, Rui Paulo wrote: > On 7 Sep 2010, at 19:41, John Nielsen wrote: > >> I am working on a network scenario which would benefit greatly from the = MIMO features and higher bandwidth of 802.11n. It's my understanding that 1= 1n is not fully supported in FreeBSD since there is no appropriate rate con= trol algorithm in the tree. Is that still the case? > > I've worked on supporting 11n on ath_rate_sample but it's incomplete. > >> >> I would _really_ like to run FreeBSD for this project, and I believe the= Atheros wireless cards I plan to use are supported by ath(4). I'd like to = find out what else needs to happen to complete the picture. I may even go s= o far as to write some code myself. :) >> >> Is anyone working on this at the moment? > > Not really, I did some work in the past, but it's incomplete. > >> >> Is it just the rate control that needs to be done or are there other par= ts involved? Is MIMO separate? > > We have MIMO on some non-Atheros drivers, but one of these drivers (Ralin= k 11n) is not yet in the tree. > >> >> Is there a detailed description of the missing pieces somewhere? Or a no= t-very-detailed summary of where to look and what to read to get started? > > Not really. There's some interest from other FreeBSD committers to get th= is going, so I'll let them chime in. > > Regards, > -- > Rui Paulo > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Sep 15 04:46:01 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE85E1065672; Wed, 15 Sep 2010 04:46:01 +0000 (UTC) (envelope-from sfourman@gmail.com) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id 89E2F8FC1D; Wed, 15 Sep 2010 04:46:01 +0000 (UTC) Received: by qyk31 with SMTP id 31so3811408qyk.13 for ; Tue, 14 Sep 2010 21:46:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=lBeIR9CaQaa7oov299thss8smts7xaxxXAcf88X8s6Q=; b=hu02Knq6ES0QOWrjMyzYRhfrdB3IUvKVVZEOx6ZGosIy+/gAGzAC56cSW+7bhUVWVz obAELTUiZMWI+QlvVCO/7uTi0aB5ot3bIEDemxXyR+e9/XMB1HraVaVNfPy2hFIOxrm8 /m8mR7tzkJj8E6FVRpzQ7ETbIyGfAA26wQsyQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=IbfgOw7gRzUB/N1hFq+1P9Mo51I3c0OER6JNfaOBKPzpwofdTzRbZw2Tkrs/p9H9cB WxcotcglUiKm42L1h0I1MdxHOEt7c8JZeN9MwNq98iND4hDk45gbYd7M6cBfzD5KGKj/ MWmVAVlhTRpqltGg7sucoiaxCBGCKa7nqZB4E= MIME-Version: 1.0 Received: by 10.224.88.81 with SMTP id z17mr683847qal.109.1284525960582; Tue, 14 Sep 2010 21:46:00 -0700 (PDT) Received: by 10.229.88.211 with HTTP; Tue, 14 Sep 2010 21:46:00 -0700 (PDT) In-Reply-To: References: <1F563B4F-B1DF-4829-966D-63264EC8AB61@jnielsen.net> <9EB9CCE4-2F4A-4653-BF11-066E3490F3AF@FreeBSD.org> Date: Tue, 14 Sep 2010 23:46:00 -0500 Message-ID: From: "Sam Fourman Jr." To: Rui Paulo Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net@freebsd.org, John Nielsen Subject: Re: Is 802.11n rate control being worked on? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Sep 2010 04:46:02 -0000 >> That would be interesting to look at. Is the code somewhere publicly available? Is it slated to hit the tree soon? > > There's a git repo somewhere on the net; can't remember where. This information should be in the FreeBSD forums. The driver is for the rt2860. > Here it is http://repo.or.cz/w/ralink_drivers.git I am using these drivers myself, they do work to a point. I have the hardware and time to do extensive testing, let me know if you need help. -- Sam Fourman Jr. Fourman Networks http://www.fourmannetworks.com From owner-freebsd-net@FreeBSD.ORG Wed Sep 15 06:20:05 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09F761065670; Wed, 15 Sep 2010 06:20:05 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id AD0558FC14; Wed, 15 Sep 2010 06:20:04 +0000 (UTC) Received: by iwn34 with SMTP id 34so7777469iwn.13 for ; Tue, 14 Sep 2010 23:20:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=yDxcb2x4vgDCvgeKbtDUoTYdW6ln+sxMx8yrd3lDHI0=; b=YkD4Tbm1WIHpeAxHSTBjEIQqb/+OpgPDPHS1g9r3i+DiHcu2J8YpOEz91X86xfD/le ixWUF+V/Y5H7Z84/EgRPy5hcLI5MHhnvSq7+bjUkArquiaX+xjQMoCtQscm/FGaO/Epe gKBf7rbw9BzhBr3O9ZRqo+7hlPYg9gVSSSetY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=r0GqXojma92HeQVlC6xGoULVNplaEGy4MAApVYvglzFvfvnhkWajUJm6azuN0nCFMa w6D9FW9164TX3x4UAC11kLz+dvSDhudo1ylE+mvSATbuw+zQPAs3f7ONXbqOHIXtZ5I5 28/WWukiPSE6+4p5ZHCk7Cn//eFZqxXi4fiNQ= MIME-Version: 1.0 Received: by 10.231.147.131 with SMTP id l3mr1224207ibv.74.1284531603873; Tue, 14 Sep 2010 23:20:03 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.231.156.206 with HTTP; Tue, 14 Sep 2010 23:20:03 -0700 (PDT) In-Reply-To: References: <1F563B4F-B1DF-4829-966D-63264EC8AB61@jnielsen.net> <9EB9CCE4-2F4A-4653-BF11-066E3490F3AF@FreeBSD.org> Date: Wed, 15 Sep 2010 14:20:03 +0800 X-Google-Sender-Auth: 5ZEc3M_85Wgr4Hc-j-JGVewgpNU Message-ID: From: Adrian Chadd To: "Sam Fourman Jr." Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net@freebsd.org, John Nielsen , Rui Paulo Subject: Re: Is 802.11n rate control being worked on? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Sep 2010 06:20:05 -0000 For 11n? adrian On 15 September 2010 12:46, Sam Fourman Jr. wrote: >>> That would be interesting to look at. Is the code somewhere publicly available? Is it slated to hit the tree soon? >> >> There's a git repo somewhere on the net; can't remember where. This information should be in the FreeBSD forums. The driver is for the rt2860. >> > Here it is > > http://repo.or.cz/w/ralink_drivers.git > I am using these drivers myself, they do work to a point. > > I have the hardware and time to do extensive testing, let me know if > you need help. > -- > > Sam Fourman Jr. > Fourman Networks > http://www.fourmannetworks.com > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Sep 15 06:44:29 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A5851065675 for ; Wed, 15 Sep 2010 06:44:29 +0000 (UTC) (envelope-from jhay@meraka.csir.co.za) Received: from zibbi.meraka.csir.co.za (unknown [IPv6:2001:4200:7000:2::1]) by mx1.freebsd.org (Postfix) with ESMTP id 629208FC1D for ; Wed, 15 Sep 2010 06:44:27 +0000 (UTC) Received: by zibbi.meraka.csir.co.za (Postfix, from userid 3973) id 943DF39824; Wed, 15 Sep 2010 08:44:24 +0200 (SAST) Date: Wed, 15 Sep 2010 08:44:24 +0200 From: John Hay To: Jack Vogel Message-ID: <20100915064424.GA31262@zibbi.meraka.csir.co.za> References: <20100723074047.GA47514@zibbi.meraka.csir.co.za> <20100820140421.GA54097@zibbi.meraka.csir.co.za> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org Subject: Re: packet loss on ixgbe using vlans and routing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Sep 2010 06:44:29 -0000 Hi, Just pinging again. I would really like to get these cards working. If there is anything I can help with... testing etc... In the meantime I have upgraded to the latest 8-stable, but the problems are still there. Thanks John On Fri, Aug 20, 2010 at 10:39:47AM -0700, Jack Vogel wrote: > OK, am setting up the hardware to look into this John. > > Jack > > > On Fri, Aug 20, 2010 at 7:04 AM, John Hay wrote: > > > Hi Jack, > > > > Have you had a chance to look at it yet? I would love to get these > > network cards working. :-) > > > > John > > > > On Fri, Jul 23, 2010 at 01:36:10AM -0700, Jack Vogel wrote: > > > Yes, I am here, I have been reading this, but I am also very busy with a > > > couple of things, please be patient, I will get on this asap. > > > > > > Cheers, > > > > > > Jack > > > > > > > > > On Fri, Jul 23, 2010 at 12:40 AM, John Hay wrote: > > > > > > > Hi, > > > > > > > > (Jack any chance that you can look at this please?) > > > > > > > > It looks like there are 2 problems with the ixgbe driver on FreeBSD-8. > > > > I have a Dell T710 with 4 X 10G ethernet interfaces (2 X Dual port > > Intel > > > > 82599 cards). It is running FreeBSD RELENG_8. > > > > > > > > 1 - When routing (using vlans) there is heavy packet loss that go away > > > > when you do "ifconfig ix2 -rxcsum". The packet loss seems to be on the > > > > receive side because I do not see them on the receiving interface with > > > > tcpdump. This seems to impact both ipv4 and ipv6. > > > > > > > > My test setup is the Dell T710 with its ix2 connected to a 10G port of > > > > a Nortel 4526GTX. On that port I have 2 vlans configured with half of > > > > the 1G ports in the one vlan and the other half in the other vlan. > > > > > > > > If I test with iperf from one of the machines on a 1G port to the T710, > > > > I get 920Mbit/s. If I do it simultaneously from a few machines > > connected > > > > to the 1G ports, all of them basically saturate their 1G links. > > > > > > > > If I now try to route from the one vlan to the other, ie. doing an > > iperf > > > > from a 1G connected machine, through the T710, to another 1G connected > > > > machine, I see packet loss, sometimes iperf is only able to do > > 100kbits/s. > > > > (Configuring a tcp relay, like socat, on the T710, and working through > > it, > > > > I again get 900Mbit/s and more.) > > > > > > > > So it seems that as long as the T710 with the 10G card is the start or > > > > end point of the connection, I get no packet loss, but as soon as it > > > > has to route, something go wrong. > > > > > > > > 2 - I see packet loss (0 - 40%) on IPv6 packets in vlans, when the > > > > machine is not the originator of the packets. This happen even with > > > > the "ifconfig ix2 -rxcsum". > > > > > > > > Let me try to describe a little more. If a neigbouring machine ping6 > > it, > > > > there will be packet loss. If it act as a router for ipv6, there will > > be > > > > packet loss. This happen even when the network is pretty idle and with > > > > different switches (Nortel and Cisco equipment). The packet loss is > > > > very fluctuating. Pinging 1000 packets might loose 1% one time and the > > > > next time 30%. Looking with tcpdump, I can see the packets arriving and > > > > going out, but the packet never arrive at the next machine. (My feeling > > is > > > > that they get lost inside the card.) The error counters on the switch > > > > does not increment. > > > > > > > > I do not see packet loss if the machine originate the packets, for > > example > > > > ping6 from the machine. Also ipv4 packets do not have any packets loss. > > If > > > > I do not use vlans, I don't see packet loss with ipv6 either. > > > > > > > > The machine also have bce 1G interfaces and I do not see the packet > > loss > > > > on them. > > > > > > > > Here is some info about the machine / setup. The numbers are pretty low > > > > because I rebooted after compiling a kernel with IPFIREWALL, > > ROUTETABLES, > > > > MROUTING and FLOWTABLE removed. I'll add my kernel config file with > > empty > > > > and commented out lines removed. > > > > > > > > pciconf -lvc > > > > ix0@pci0:129:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 > > > > rev=0x01 hdr=0x00 > > > > vendor = 'Intel Corporation' > > > > class = network > > > > subclass = ethernet > > > > cap 01[40] = powerspec 3 supports D0 D3 current D0 > > > > cap 05[50] = MSI supports 1 message, 64 bit, vector masks > > > > cap 11[70] = MSI-X supports 64 messages in map 0x20 enabled > > > > cap 10[a0] = PCI-Express 2 endpoint max data 256(512) link x8(x8) > > > > ix1@pci0:129:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 > > > > rev=0x01 hdr=0x00 > > > > vendor = 'Intel Corporation' > > > > class = network > > > > subclass = ethernet > > > > cap 01[40] = powerspec 3 supports D0 D3 current D0 > > > > cap 05[50] = MSI supports 1 message, 64 bit, vector masks > > > > cap 11[70] = MSI-X supports 64 messages in map 0x20 enabled > > > > cap 10[a0] = PCI-Express 2 endpoint max data 256(512) link x8(x8) > > > > ix2@pci0:131:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 > > > > rev=0x01 hdr=0x00 > > > > vendor = 'Intel Corporation' > > > > class = network > > > > subclass = ethernet > > > > cap 01[40] = powerspec 3 supports D0 D3 current D0 > > > > cap 05[50] = MSI supports 1 message, 64 bit, vector masks > > > > cap 11[70] = MSI-X supports 64 messages in map 0x20 enabled > > > > cap 10[a0] = PCI-Express 2 endpoint max data 256(512) link x8(x8) > > > > ix3@pci0:131:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 > > > > rev=0x01 hdr=0x00 > > > > vendor = 'Intel Corporation' > > > > class = network > > > > subclass = ethernet > > > > cap 01[40] = powerspec 3 supports D0 D3 current D0 > > > > cap 05[50] = MSI supports 1 message, 64 bit, vector masks > > > > cap 11[70] = MSI-X supports 64 messages in map 0x20 enabled > > > > cap 10[a0] = PCI-Express 2 endpoint max data 256(512) link x8(x8) > > > > > > > > output of vmstat -i > > > > > > > > interrupt total rate > > > > irq19: ehci0 28371 0 > > > > irq21: uhci2 uhci4+ 48 0 > > > > irq23: atapci0 46 0 > > > > irq34: mpt0 146954 2 > > > > cpu0: timer 112205297 1999 > > > > irq256: bce0 52063 0 > > > > irq257: bce1 1 0 > > > > irq258: bce2 1 0 > > > > irq259: bce3 1 0 > > > > irq260: ix0:que 0 142258 2 > > > > irq261: ix0:que 1 56464 1 > > > > irq262: ix0:que 2 56199 1 > > > > irq263: ix0:que 3 56198 1 > > > > irq264: ix0:que 4 66569 1 > > > > irq265: ix0:que 5 56148 1 > > > > irq266: ix0:que 6 56217 1 > > > > irq267: ix0:que 7 56311 1 > > > > irq268: ix0:que 8 56169 1 > > > > irq269: ix0:que 9 69485 1 > > > > irq270: ix0:que 10 56176 1 > > > > irq271: ix0:que 11 56205 1 > > > > irq272: ix0:que 12 56281 1 > > > > irq273: ix0:que 13 56359 1 > > > > irq274: ix0:que 14 56292 1 > > > > irq275: ix0:que 15 56197 1 > > > > irq276: ix0:link 2 0 > > > > irq277: ix1:que 0 107873 1 > > > > irq278: ix1:que 1 56094 0 > > > > irq279: ix1:que 2 56097 0 > > > > irq280: ix1:que 3 56096 0 > > > > irq281: ix1:que 4 65439 1 > > > > irq282: ix1:que 5 56091 0 > > > > irq283: ix1:que 6 56092 0 > > > > irq284: ix1:que 7 56098 0 > > > > irq285: ix1:que 8 56091 0 > > > > irq286: ix1:que 9 56096 0 > > > > irq287: ix1:que 10 56093 0 > > > > irq288: ix1:que 11 56091 0 > > > > irq289: ix1:que 12 56096 0 > > > > irq290: ix1:que 13 56095 0 > > > > irq291: ix1:que 14 57125 1 > > > > irq292: ix1:que 15 56093 0 > > > > irq293: ix1:link 1 0 > > > > irq294: ix2:que 0 231250 4 > > > > irq295: ix2:que 1 57784 1 > > > > irq296: ix2:que 2 69956 1 > > > > irq297: ix2:que 3 59498 1 > > > > irq298: ix2:que 4 58201 1 > > > > irq299: ix2:que 5 58599 1 > > > > irq300: ix2:que 6 57813 1 > > > > irq301: ix2:que 7 60075 1 > > > > irq302: ix2:que 8 68639 1 > > > > irq303: ix2:que 9 58194 1 > > > > irq304: ix2:que 10 60752 1 > > > > irq305: ix2:que 11 57628 1 > > > > irq306: ix2:que 12 66796 1 > > > > irq307: ix2:que 13 63307 1 > > > > irq308: ix2:que 14 60788 1 > > > > irq309: ix2:que 15 59102 1 > > > > irq310: ix2:link 5 0 > > > > irq311: ix3:que 0 56090 0 > > > > irq312: ix3:que 1 56090 0 > > > > irq313: ix3:que 2 56090 0 > > > > irq314: ix3:que 3 56090 0 > > > > irq315: ix3:que 4 56090 0 > > > > irq316: ix3:que 5 56090 0 > > > > irq317: ix3:que 6 56090 0 > > > > irq318: ix3:que 7 56090 0 > > > > irq319: ix3:que 8 56090 0 > > > > irq320: ix3:que 9 56090 0 > > > > irq321: ix3:que 10 56090 0 > > > > irq322: ix3:que 11 56090 0 > > > > irq323: ix3:que 12 56090 0 > > > > irq324: ix3:que 13 56090 0 > > > > irq325: ix3:que 14 56090 0 > > > > irq326: ix3:que 15 56090 0 > > > > cpu1: timer 112196134 1999 > > > > cpu10: timer 112196179 1999 > > > > cpu3: timer 112196135 1999 > > > > cpu8: timer 112196108 1999 > > > > cpu4: timer 112196161 1999 > > > > cpu11: timer 112196179 1999 > > > > cpu5: timer 112196161 1999 > > > > cpu13: timer 112196179 1999 > > > > cpu6: timer 112196161 1999 > > > > cpu14: timer 112196179 1999 > > > > cpu2: timer 112196106 1999 > > > > cpu12: timer 112196179 1999 > > > > cpu7: timer 112196161 1999 > > > > cpu9: timer 112196155 1999 > > > > cpu15: timer 112196179 1999 > > > > Total 1799390156 32072 > > > > > > > > netstat -m > > > > > > > > 133178/4042/137220 mbufs in use (current/cache/total) > > > > 133112/2062/135174/262144 mbuf clusters in use > > (current/cache/total/max) > > > > 133112/2056 mbuf+clusters out of packet secondary zone in use > > > > (current/cache) > > > > 0/20/20/131072 4k (page size) jumbo clusters in use > > > > (current/cache/total/max) > > > > 0/0/0/65536 9k jumbo clusters in use (current/cache/total/max) > > > > 0/0/0/32768 16k jumbo clusters in use (current/cache/total/max) > > > > 299518K/5214K/304733K bytes allocated to network (current/cache/total) > > > > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > > > > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > > > > 0/0/0 sfbufs in use (current/peak/max) > > > > 0 requests for sfbufs denied > > > > 0 requests for sfbufs delayed > > > > 0 requests for I/O initiated by sendfile > > > > 0 calls to protocol drain routines > > > > > > > > kernel config file, basically started with 64 bit and removed the stuff > > > > I do not need. > > > > > > > > cpu HAMMER > > > > ident SEEKAT > > > > device ipmi > > > > makeoptions DEBUG=-g # Build kernel with gdb(1) > > debug > > > > symbols > > > > options SCHED_ULE # ULE scheduler > > > > options PREEMPTION # Enable kernel thread > > preemption > > > > options INET # InterNETworking > > > > options INET6 # IPv6 communications protocols > > > > options SCTP # Stream Control Transmission > > > > Protocol > > > > options FFS # Berkeley Fast Filesystem > > > > options SOFTUPDATES # Enable FFS soft updates > > support > > > > options UFS_DIRHASH # Improve performance on big > > > > directories > > > > options CD9660 # ISO 9660 Filesystem > > > > options PROCFS # Process filesystem (requires > > > > PSEUDOFS) > > > > options PSEUDOFS # Pseudo-filesystem framework > > > > options GEOM_PART_GPT # GUID Partition Tables. > > > > options GEOM_LABEL # Provides labelization > > > > options COMPAT_43TTY # BSD 4.3 TTY compat (sgtty) > > > > options COMPAT_IA32 # Compatible with i386 binaries > > > > options COMPAT_FREEBSD4 # Compatible with FreeBSD4 > > > > options COMPAT_FREEBSD5 # Compatible with FreeBSD5 > > > > options COMPAT_FREEBSD6 # Compatible with FreeBSD6 > > > > options COMPAT_FREEBSD7 # Compatible with FreeBSD7 > > > > options KTRACE # ktrace(1) support > > > > options STACK # stack(9) support > > > > options SYSVSHM # SYSV-style shared memory > > > > options SYSVMSG # SYSV-style message queues > > > > options SYSVSEM # SYSV-style semaphores > > > > options P1003_1B_SEMAPHORES # POSIX-style semaphores > > > > options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time > > > > extensions > > > > options PRINTF_BUFR_SIZE=128 # Prevent printf output being > > > > interspersed. > > > > options KBD_INSTALL_CDEV # install a CDEV entry in /dev > > > > options HWPMC_HOOKS # Necessary kernel hooks for > > > > hwpmc(4) > > > > options INCLUDE_CONFIG_FILE # Include this file in kernel > > > > options SMP # Symmetric MultiProcessor > > Kernel > > > > device cpufreq > > > > device acpi > > > > device pci > > > > device ata > > > > device atapicd # ATAPI CDROM drives > > > > device mpt # LSI-Logic MPT-Fusion > > > > device scbus # SCSI bus (required for SCSI) > > > > device da # Direct Access (disks) > > > > device pass # Passthrough device (direct SCSI > > access) > > > > device atkbdc # AT keyboard controller > > > > device atkbd # AT keyboard > > > > device psm # PS/2 mouse > > > > device kbdmux # keyboard multiplexer > > > > device vga # VGA video card driver > > > > device splash # Splash screen and screen saver > > support > > > > device sc > > > > device agp # support several AGP chipsets > > > > device uart # Generic UART driver > > > > device loop # Network loopback > > > > device random # Entropy device > > > > device ether # Ethernet support > > > > device pty # BSD-style compatibility pseudo ttys > > > > device bpf # Berkeley packet filter > > > > device uhci # UHCI PCI->USB interface > > > > device ehci # EHCI PCI->USB interface (USB 2.0) > > > > device usb # USB Bus (required) > > > > device uhid # "Human Interface Devices" > > > > device ukbd # Keyboard > > > > device umass # Disks/Mass storage - Requires scbus > > and > > > > da > > > > device ums # Mouse > > > > > > > > kldstat > > > > Id Refs Address Size Name > > > > 1 55 0xffffffff80100000 6ea290 kernel > > > > 2 1 0xffffffff807eb000 19e088 zfs.ko > > > > 3 2 0xffffffff8098a000 3860 opensolaris.ko > > > > 4 2 0xffffffff8098e000 20448 krpc.ko > > > > 5 1 0xffffffff809af000 21100 geom_mirror.ko > > > > 6 1 0xffffffff809d1000 66c0 if_vlan.ko > > > > 7 1 0xffffffff809d8000 506c8 if_bce.ko > > > > 8 2 0xffffffff80a29000 3ec20 miibus.ko > > > > 9 1 0xffffffff80a68000 243e0 if_ixgbe.ko > > > > 10 1 0xffffffff80a8d000 1e08 coretemp.ko > > > > > > > > ifconfig ix2 (with -rxcsum and global addrs modified) > > > > ix2: flags=8843 metric 0 mtu > > 1500 > > > > > > options=5b8 > > > > ether 00:1b:21:57:ef:7c > > > > inet6 fe80::21b:21ff:fe57:ef7c%ix2 prefixlen 64 scopeid 0x3 > > > > nd6 options=3 > > > > media: Ethernet autoselect (10Gbase-SR ) > > > > status: active > > > > ifconfig ix2.1 > > > > ix2.1: flags=8843 metric 0 mtu > > 1500 > > > > ether 00:1b:21:57:ef:7c > > > > inet 10.0.28.2 netmask 0xffffff00 broadcast 10.0.28.255 > > > > inet6 fe80::21b:21ff:fe57:b420%ix2.1 prefixlen 64 scopeid 0x9 > > > > inet6 2001:0:0:3:21b:21ff:fe57:b420 prefixlen 64 > > > > inet6 2001:0:0:3:: prefixlen 64 anycast > > > > nd6 options=3 > > > > media: Ethernet autoselect (10Gbase-SR ) > > > > status: active > > > > vlan: 1 parent interface: ix2 > > > > ifconfig ix2.8 > > > > ix2.8: flags=8843 metric 0 mtu > > 1500 > > > > ether 00:1b:21:57:ef:7c > > > > inet 10.0.8.50 netmask 0xffffff00 broadcast 10.0.8.255 > > > > inet6 fe80::21b:21ff:fe57:b420%ix2.8 prefixlen 64 scopeid 0xa > > > > inet6 2001:0:0:1:21b:21ff:fe57:b420 prefixlen 64 > > > > inet6 2001:0:0:1:: prefixlen 64 anycast > > > > nd6 options=3 > > > > media: Ethernet autoselect (10Gbase-SR ) > > > > status: active > > > > vlan: 8 parent interface: ix2 > > > > > > > > John > > > > -- > > > > John Hay -- jhay@meraka.csir.co.za / jhay@FreeBSD.org From owner-freebsd-net@FreeBSD.ORG Wed Sep 15 15:20:06 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA673106564A; Wed, 15 Sep 2010 15:20:06 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.cksoft.de (mail.cksoft.de [IPv6:2001:4068:10::3]) by mx1.freebsd.org (Postfix) with ESMTP id 840EF8FC0C; Wed, 15 Sep 2010 15:20:06 +0000 (UTC) Received: from localhost (amavis.fra.cksoft.de [192.168.74.71]) by mail.cksoft.de (Postfix) with ESMTP id D6B4941C7AD; Wed, 15 Sep 2010 17:20:05 +0200 (CEST) X-Virus-Scanned: amavisd-new at cksoft.de Received: from mail.cksoft.de ([192.168.74.103]) by localhost (amavis.fra.cksoft.de [192.168.74.71]) (amavisd-new, port 10024) with ESMTP id aRXBqrc5MKsX; Wed, 15 Sep 2010 17:20:05 +0200 (CEST) Received: by mail.cksoft.de (Postfix, from userid 66) id 5B12441C76D; Wed, 15 Sep 2010 17:20:05 +0200 (CEST) Received: from maildrop.int.zabbadoz.net (maildrop.int.zabbadoz.net [10.111.66.10]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.int.zabbadoz.net (Postfix) with ESMTP id B40CE4448F3; Wed, 15 Sep 2010 15:19:50 +0000 (UTC) Date: Wed, 15 Sep 2010 15:19:50 +0000 (UTC) From: "Bjoern A. Zeeb" X-X-Sender: bz@maildrop.int.zabbadoz.net To: Andre Oppermann In-Reply-To: <4C8E0C1E.2020707@networx.ch> Message-ID: <20100915151632.E31898@maildrop.int.zabbadoz.net> References: <4C8E0C1E.2020707@networx.ch> X-OpenPGP-Key: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org Subject: Re: TCP loopback socket fusing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Sep 2010 15:20:06 -0000 On Mon, 13 Sep 2010, Andre Oppermann wrote: Hey, > When a TCP connection via loopback back to localhost is made the whole > send, segmentation and receive path (with larger packets though) is still > executed. This has some considerable overhead. > > To short-circuit the send and receive sockets on localhost TCP connections > I've made a proof-of-concept patch that directly places the data in the > other side's socket buffer without doing any packetization and other protocol > overhead (like UNIX domain sockets). The connections setup (SYN, SYN-ACK, > ACK) and shutdown are still handled by normal TCP segments via loopback so > that firewalling stills works. The actual payload data during the session > won't be seen and the sequence numbers don't move other than for SYN and FIN. > The sequence are remain valid though. Obviously tcpdump won't see any data > transfers either if the connection has fused sockets. > > Preliminary testing (with WITNESS and INVARIANTS enabled) has shown stable > operation and a rough doubling of the throughput on loopback connections. > I've tested most socket teardown cases and it behaves fine. I'm not entirely > sure I've got all possible path's but the way it is integrated should > properly > defuse the sockets in all situations. Three comments in reverse order: 1 If S/S+A/A and shutdown aren't shortcut, can you always rely on proper payload order, especially in the shutdown case? 2 Given my experience with epairs, which are basically a loop with two interfaces and even interface queues, any significant delay you are seeing is _not_ due to longer code paths through the stack but simply because of the netisr. 3 If properly doing this for TCP, we should probably also do it for other protocols. /bz -- Bjoern A. Zeeb Welcome a new stage of life. From owner-freebsd-net@FreeBSD.ORG Wed Sep 15 15:48:08 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0633A106566C for ; Wed, 15 Sep 2010 15:48:08 +0000 (UTC) (envelope-from oppermann@networx.ch) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 521098FC13 for ; Wed, 15 Sep 2010 15:48:06 +0000 (UTC) Received: (qmail 72458 invoked from network); 15 Sep 2010 15:42:53 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 15 Sep 2010 15:42:53 -0000 Message-ID: <4C90EAB7.2000902@networx.ch> Date: Wed, 15 Sep 2010 17:48:07 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: "Bjoern A. Zeeb" References: <4C8E0C1E.2020707@networx.ch> <20100915151632.E31898@maildrop.int.zabbadoz.net> In-Reply-To: <20100915151632.E31898@maildrop.int.zabbadoz.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org Subject: Re: TCP loopback socket fusing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Sep 2010 15:48:08 -0000 On 15.09.2010 17:19, Bjoern A. Zeeb wrote: > On Mon, 13 Sep 2010, Andre Oppermann wrote: > > Hey, > >> When a TCP connection via loopback back to localhost is made the whole >> send, segmentation and receive path (with larger packets though) is still >> executed. This has some considerable overhead. >> >> To short-circuit the send and receive sockets on localhost TCP connections >> I've made a proof-of-concept patch that directly places the data in the >> other side's socket buffer without doing any packetization and other protocol >> overhead (like UNIX domain sockets). The connections setup (SYN, SYN-ACK, >> ACK) and shutdown are still handled by normal TCP segments via loopback so >> that firewalling stills works. The actual payload data during the session >> won't be seen and the sequence numbers don't move other than for SYN and FIN. >> The sequence are remain valid though. Obviously tcpdump won't see any data >> transfers either if the connection has fused sockets. >> >> Preliminary testing (with WITNESS and INVARIANTS enabled) has shown stable >> operation and a rough doubling of the throughput on loopback connections. >> I've tested most socket teardown cases and it behaves fine. I'm not entirely >> sure I've got all possible path's but the way it is integrated should properly >> defuse the sockets in all situations. > > Three comments in reverse order: > > 1 If S/S+A/A and shutdown aren't shortcut, can you always rely on proper > payload order, especially in the shutdown case? Yes. The payload is always directly placed in the receive socket buffer of the other socket, never in the send buffer. There is never any unsent data left in the send buffer that could become reordered. > 2 Given my experience with epairs, which are basically a loop with two > interfaces and even interface queues, any significant delay you are > seeing is _not_ due to longer code paths through the stack but > simply because of the netisr. I haven't measured delay, only bandwidth. And that's with WITNESS and INVARIANTS enabled. You are probably right, the netisr is taking its toll. Especially the TCP_INFO lock may have some contention in the loopback case on SMP. Though a lot of mbuf allocations, packet manipulations and instructions (instruction cache) are avoided by fusing the sockets together. > 3 If properly doing this for TCP, we should probably also do it for > other protocols. UNIX domain sockets already do this. This implementation is particular for TCP and only touches the protocol specific parts. It's not done at the socket layer. For UDP it's not that easy to do as most UDP connections are one-off packets and no permanent binding between two sockets exists. For SCTP I don't know. From glancing over the code it seems they have, at least partially, their own socket buffer code. How difficult a fused socket there would be I can't say. -- Andre From owner-freebsd-net@FreeBSD.ORG Wed Sep 15 15:56:52 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F6D7106566C for ; Wed, 15 Sep 2010 15:56:52 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 4BD9B8FC1F for ; Wed, 15 Sep 2010 15:56:50 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA11317; Wed, 15 Sep 2010 18:38:17 +0300 (EEST) (envelope-from avg@freebsd.org) Message-ID: <4C90E869.8000400@freebsd.org> Date: Wed, 15 Sep 2010 18:38:17 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100909 Lightning/1.0b2 Thunderbird/3.1.3 MIME-Version: 1.0 To: Steven Hartland References: <5DB6E7C798E44D33A05673F4B773405E@multiplay.co.uk><4C85E91E.1010602@icyb.net.ua><4C873914.40404@freebsd.org><20100908084855.GF2465@deviant.kiev.zoral.com.ua><4C874F00.3050605@freebsd.org><4C8D087B.5040404@freebsd.org><03537796FAB54E02959E2D64FC83004F@multiplay.co.uk><4C8D280F.3040803@freebsd.org><3FBF66BF11AA4CBBA6124CA435A4A31B@multiplay.co.uk><4C8E4212.30000@freebsd.org> <4C90B4C8.90203@freebsd.org> <6DFACB27CA8A4A22898BC81E55C4FD36@multiplay.co.uk> <4C90D3A1.7030008@freebsd.org> <0B1A90A08DFE4ADA9540F9F3846FDF38@multiplay.co.uk> <4C90E328.20606@freebsd.org> In-Reply-To: <4C90E328.20606@freebsd.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, jhell , Pawel Jakub Dawidek , freebsd-net@freebsd.org Subject: Re: zfs very poor performance compared to ufs due to lack of cache? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Sep 2010 15:56:52 -0000 on 15/09/2010 18:15 Andriy Gapon said the following: > on 15/09/2010 18:04 Steven Hartland said the following: >> Hmm, so taking a different track on the issue is the a way to make sendfile use data >> directly from ARC instead of having to copy it first? > > Well, theoretically everything is possible, but I am not sure if it's feasible. > It's a lot of work anyways, it should be a very specialized sendfile and a lot if > inter-layer knowledge and dependencies. > Don't hold your breath for it. Perhaps some middle-ground solution can be developed with less effort. This solution would be specific to filesystems that don't use buffer cache, so it wouldn't touch any pages, but instead it would use regular VOP_READ into a mbuf. So, there would be copying, but page caches won't be unnecessarily "polluted" with second copy of the data and this all would happen in kernel giving an advantage over userland solution with read(2)+send(2). Having said that, I see that OpenSolaris has a mechanism for something like that. The mechanism can either globally enabled or enabled for file over certain size. The mechanism uses dedicated kernel threads that get data using direct I/O, buffer and send it. That's my impression from a quick look, I may have gotten things wrong. -- Andriy Gapon From owner-freebsd-net@FreeBSD.ORG Wed Sep 15 16:00:33 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BBAB11065670 for ; Wed, 15 Sep 2010 16:00:33 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 216D28FC18 for ; Wed, 15 Sep 2010 16:00:32 +0000 (UTC) Received: (qmail 72596 invoked from network); 15 Sep 2010 15:55:19 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 15 Sep 2010 15:55:19 -0000 Message-ID: <4C90EDA1.6020501@freebsd.org> Date: Wed, 15 Sep 2010 18:00:33 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 MIME-Version: 1.0 To: Andriy Gapon References: <5DB6E7C798E44D33A05673F4B773405E@multiplay.co.uk><4C85E91E.1010602@icyb.net.ua><4C873914.40404@freebsd.org><20100908084855.GF2465@deviant.kiev.zoral.com.ua><4C874F00.3050605@freebsd.org><4C8D087B.5040404@freebsd.org><03537796FAB54E02959E2D64FC83004F@multiplay.co.uk><4C8D280F.3040803@freebsd.org><3FBF66BF11AA4CBBA6124CA435A4A31B@multiplay.co.uk><4C8E4212.30000@freebsd.org> <4C90B4C8.90203@freebsd.org> <6DFACB27CA8A4A22898BC81E55C4FD36@multiplay.co.uk> <4C90D3A1.7030008@freebsd.org> <0B1A90A08DFE4ADA9540F9F3846FDF38@multiplay.co.uk> <4C90E328.20606@freebsd.org> <4C90E869.8000400@freebsd.org> In-Reply-To: <4C90E869.8000400@freebsd.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, jhell , Steven Hartland , Pawel Jakub Dawidek , freebsd-net@freebsd.org Subject: Re: zfs very poor performance compared to ufs due to lack of cache? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Sep 2010 16:00:33 -0000 On 15.09.2010 17:38, Andriy Gapon wrote: > on 15/09/2010 18:15 Andriy Gapon said the following: >> on 15/09/2010 18:04 Steven Hartland said the following: >>> Hmm, so taking a different track on the issue is the a way to make sendfile use data >>> directly from ARC instead of having to copy it first? >> >> Well, theoretically everything is possible, but I am not sure if it's feasible. >> It's a lot of work anyways, it should be a very specialized sendfile and a lot if >> inter-layer knowledge and dependencies. >> Don't hold your breath for it. > > Perhaps some middle-ground solution can be developed with less effort. > This solution would be specific to filesystems that don't use buffer cache, so it > wouldn't touch any pages, but instead it would use regular VOP_READ into a mbuf. > So, there would be copying, but page caches won't be unnecessarily "polluted" with > second copy of the data and this all would happen in kernel giving an advantage > over userland solution with read(2)+send(2). Is there a quick way of deciding within sendfile(2) whether a file resides on a filesystem that doesn't use the buffer cache? > Having said that, I see that OpenSolaris has a mechanism for something like that. > The mechanism can either globally enabled or enabled for file over certain size. > The mechanism uses dedicated kernel threads that get data using direct I/O, buffer > and send it. > That's my impression from a quick look, I may have gotten things wrong. -- Andre From owner-freebsd-net@FreeBSD.ORG Wed Sep 15 16:31:53 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 03B151065670; Wed, 15 Sep 2010 16:31:53 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id D75928FC12; Wed, 15 Sep 2010 16:31:51 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA12300; Wed, 15 Sep 2010 19:31:50 +0300 (EEST) (envelope-from avg@freebsd.org) Message-ID: <4C90F4F6.209@freebsd.org> Date: Wed, 15 Sep 2010 19:31:50 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100909 Lightning/1.0b2 Thunderbird/3.1.3 MIME-Version: 1.0 To: Andre Oppermann References: <5DB6E7C798E44D33A05673F4B773405E@multiplay.co.uk><4C85E91E.1010602@icyb.net.ua><4C873914.40404@freebsd.org><20100908084855.GF2465@deviant.kiev.zoral.com.ua><4C874F00.3050605@freebsd.org><4C8D087B.5040404@freebsd.org><03537796FAB54E02959E2D64FC83004F@multiplay.co.uk><4C8D280F.3040803@freebsd.org><3FBF66BF11AA4CBBA6124CA435A4A31B@multiplay.co.uk><4C8E4212.30000@freebsd.org> <4C90B4C8.90203@freebsd.org> <6DFACB27CA8A4A22898BC81E55C4FD36@multiplay.co.uk> <4C90D3A1.7030008@freebsd.org> <0B1A90A08DFE4ADA9540F9F3846FDF38@multiplay.co.uk> <4C90E328.20606@freebsd.org> <4C90E869.8000400@freebsd.org> <4C90EDA1.6020501@freebsd.org> In-Reply-To: <4C90EDA1.6020501@freebsd.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-net@freebsd.org Subject: Re: zfs very poor performance compared to ufs due to lack of cache? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Sep 2010 16:31:53 -0000 on 15/09/2010 19:00 Andre Oppermann said the following: > Is there a quick way of deciding within sendfile(2) whether a file resides > on a filesystem that doesn't use the buffer cache? I don't know of any reliable way to do it. -- Andriy Gapon From owner-freebsd-net@FreeBSD.ORG Wed Sep 15 16:45:09 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0991D1065670 for ; Wed, 15 Sep 2010 16:45:09 +0000 (UTC) (envelope-from gljennjohn@googlemail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 861E18FC18 for ; Wed, 15 Sep 2010 16:45:08 +0000 (UTC) Received: by bwz15 with SMTP id 15so875648bwz.13 for ; Wed, 15 Sep 2010 09:45:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:cc:subject :message-id:in-reply-to:references:reply-to:x-mailer:mime-version :content-type:content-transfer-encoding; bh=cx9Hl1OShTSbuxmVoRQ8jDbY7VUdNE4vou7sn97CHqI=; b=cpaxnw8zog9uD/y2KumIDtfM//QAiV7cQdSlZIEuy6Bd6XtwrOs6TBIIygF+/tZily NUQQUpGfTOrt0lUyFX9YL8tSadVi3bnA/qtIPy4lPSts2cGju30PBNLWWMcyWOU/dxi9 xE8jEf9Bsmta3cS+6bZ/bik1XcMQjdMZIcwyU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :x-mailer:mime-version:content-type:content-transfer-encoding; b=CgaGcwnuCDCIabXnvFPqEShfATcdJzF0C7rmQYgQW3k7/RvxDYnIsP1qJpmYWX0KkX 8Ih7Y+5ZzFlW+k5iUzP4okdhChdEPXYOPJt14PE+LZKAoz6/uOPb+L+jq+vda6Ld2BS/ d/aa6SBOwM3Us4DPA8OdZBbkH0kfSWpRiYkvI= Received: by 10.204.77.212 with SMTP id h20mr1498998bkk.33.1284567274487; Wed, 15 Sep 2010 09:14:34 -0700 (PDT) Received: from ernst.jennejohn.org (p578E15EB.dip.t-dialin.net [87.142.21.235]) by mx.google.com with ESMTPS id d27sm1494439bku.22.2010.09.15.09.14.32 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 15 Sep 2010 09:14:33 -0700 (PDT) Date: Wed, 15 Sep 2010 18:14:31 +0200 From: Gary Jennejohn To: Andre Oppermann Message-ID: <20100915181431.30677523@ernst.jennejohn.org> In-Reply-To: <20100915151632.E31898@maildrop.int.zabbadoz.net> References: <4C8E0C1E.2020707@networx.ch> <20100915151632.E31898@maildrop.int.zabbadoz.net> X-Mailer: Claws Mail 3.7.6 (GTK+ 2.18.7; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org Subject: Re: TCP loopback socket fusing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gljennjohn@googlemail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Sep 2010 16:45:09 -0000 On Mon, 13 Sep 2010, Andre Oppermann wrote: > Preliminary testing (with WITNESS and INVARIANTS enabled) has shown stable > operation and a rough doubling of the throughput on loopback connections. > I've tested most socket teardown cases and it behaves fine. I'm not entirely > sure I've got all possible path's but the way it is integrated should > properly defuse the sockets in all situations. > I tried this out with the following results: a) booted the new kernel, started X, started firefox -> hard hang, had to reset the box to recover. Note that firefox uses wwwoffle as a local, caching proxy and wwwwoffle is accessed through localhost:8080 b) tried (a) again to make sure it wasn't a fluke -> same result c) booted anew but started opera instead, which does _not_ use wwwoffle as its proxy (net.inet.tcp.loopfuse=1) -> OK d) I then set net.inet.tcp.loopfuse=0 before starting firefox -> OK e) set net.inet.tcp.loopfuse=1 and ran cvsup to update my CVS tree followed by checking out the changed sources, which uses loopback to talk to cvsupd -> OK So, somewhow trying to access wwwoffle through localhost:8080 causes a hard hang of the box. Whether this has something to do with the port number or just strange behavior on the part of wwwoffle I can't say, because the hard hang makes debugging impossible. By hard hang I mean that there is no visible activity, gkrellm isn't updating, mouse and keyboard are ignored and ping from a different machine shows no reaction, so I'd say the kernel is pretty much wedged. For now I'm setting net.inet.tcp.loopfuse=0 in /etc/sysctl.conf. -- Gary Jennejohn From owner-freebsd-net@FreeBSD.ORG Thu Sep 16 14:28:37 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2CA1C106566C for ; Thu, 16 Sep 2010 14:28:37 +0000 (UTC) (envelope-from vl.varlog@gmail.com) Received: from mail-ew0-f54.google.com (mail-ew0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id A73738FC14 for ; Thu, 16 Sep 2010 14:28:36 +0000 (UTC) Received: by ewy22 with SMTP id 22so712301ewy.13 for ; Thu, 16 Sep 2010 07:28:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:date:from:x-priority :message-id:to:subject:mime-version:content-type :content-transfer-encoding; bh=x9V7EyqERXgKsF2CbVjlXUmFS3rU/scegGcqtqdp0d4=; b=x1l8qWIDmkz5zUorsDec58VUlVRNWAMEjxzNhVsgBMMiGtzVScpWGAkwnhTYW+NaRx J4EIpM7yWHTjHvCYhrrhEvwQwhp0Q88CawVvTF+Uv0Kbprhanln4wsLf9W2MH9bvaEmF cmYgPXnc8upbHDKtyKY45L2430774dwxRcu2w= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:x-priority:message-id:to:subject:mime-version :content-type:content-transfer-encoding; b=Km3FGo1ZGoZjxkSKCvK83PGZjL+IL59A5Wpk466pdQGnnUtVpPTm0Xl70vDUmQbmIt yAOjHLrxzvhEvZAuReDXdOe6CXQMJDAH0A/4Evxj6/kf99xXnkC8tiF+uUQAwb6npMVH o119bouxXs3gKtCFhiIZ9hEXh9ScEXiNSfi2Y= Received: by 10.213.33.194 with SMTP id i2mr2470914ebd.10.1284645710928; Thu, 16 Sep 2010 07:01:50 -0700 (PDT) Received: from v-grigorov-xp.mail.msk ([195.218.191.171]) by mx.google.com with ESMTPS id v59sm3949869eeh.16.2010.09.16.07.01.46 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 16 Sep 2010 07:01:47 -0700 (PDT) Date: Thu, 16 Sep 2010 18:00:53 +0400 From: Vladimir Grigorov X-Priority: 3 (Normal) Message-ID: <273436110.20100916180053@gmail.com> To: freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Subject: Strange FreeBSD behavior when trying to forward beetween ipsec crypted gif's. May be a problem with ICMP unreach packets at all X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 14:28:37 -0000 Greetings all. I have strange problems related to passage icmp need-frag packets, and, as = result, all packets with packets length greater than output gif MTU. Network diagram: [HostA] -- (mtu 1500) --- [FW1] --- ipsec gif mtu 1280 <-gif1 -- [FW2] - gi= f0 -> ipsec gif mtu 6100 - [FW3] -(mtu 1500) - [HostB] All FW's - Freebsd hosts HostA - freebsd host HostB - Cisco 3750e switch in L3 mode HostA can reach HostB and vice versa. Ping with length above 1280 works fin= e (pmtu =3D 1280). Ping with len=3D1281 without df bit also work fine. But = ping with mtu 1281 fails.=20 Question: Why FW2 does not send ICMP need-fragment-but-DF-set message to Ho= stB ?=20 I try to permit icmp from all interfaces on FW2, explicit send unreachable = packet for all ip packets from defined source ip - nothing happens. I see i= ncreased packets counts related my source ip, but cant see responce icmps w= ith unreachable code uname -a FreeBSD fw2-mru.astrum-nival.com 8.0-RELEASE-p3 FreeBSD 8.0-RELEASE-p3 #3: = Thu Jul 1 18:24:35 MSD 2010 root@fw2-mru.astrum-nival.com:/usr/obj/usr= /src/sys/gw amd64 ifconfig gif0 gif0: flags=3D8051 metric 0 mtu 6100 tunnel inet 217.69.143.28 --> 217.69.143.57 inet 10.192.224.5 --> 10.192.224.6 netmask 0xfffffffc=20 options=3D1 ifconfig gif1 gif1: flags=3D8051 metric 0 mtu 1280 tunnel inet 217.69.143.28 --> 88.212.205.166 inet 10.160.192.6 --> 10.160.192.5 netmask 0xfffffffc=20 options=3D1 netstat -nr | grep 192.168.224 192.168.224.0/22 10.192.224.6 UG1 0 36031303 gif0 netstat -nr | grep 192.168.160. 192.168.160.0/24 10.160.192.5 UG1 0 10969867 gif1 # ipfw show 00006 10 6505 allow icmp from any to 192.168.225.1 via= gif0 00100 10524445 1225052712 allow ip from any to any via lo0 00200 0 0 deny ip from any to 127.0.0.0/8 00300 0 0 deny ip from 127.0.0.0/8 to any 00305 2054 433651 allow icmp from any to any via gif0 icmp= types 3,11 00306 0 0 allow icmp from any to 192.168.225.1 via= gif0 00310 6960 575159 nat 220 ip from table(10) to any via vla= n220 00315 1198 70832 deny ip from not me to 192.168.66.0/23 o= ut xmit vlan220 00320 6512 1611912 nat 220 ip from 192.168.66.0/23 to 192.1= 68.13.199 in recv vlan220 00400 114560294 8963623578 nat 123 ip from 192.168.196.0/24 to any = out via vlan506 00402 36831424 2199804860 nat 123 ip from 192.168.193.0/24 to any = out via vlan506 00403 153380 9265905 nat 123 ip from 192.168.197.0/24 to any = out via vlan506 00500 0 0 nat 123 ip from any to 195.211.130.9 in = via vlan506 00501 147593882 174870597871 nat 123 ip from any to 195.211.130.9 in = via vlan500 01100 0 0 allow tcp from table(21) to table(23) ds= t-port 29000 01110 0 0 deny tcp from table(22) to table(23) dst= -port 29000 01120 3 144 deny tcp from table(20) to table(23) dst= -port 29000 65530 589120438508 133855063718386 allow ip from any to any 65535 0 0 deny ip from any to any try to ping from cisco: c3750e.gldn#ping 192.168.160.248 source 192.168.225.1 repea 5 size 1281 df Type escape sequence to abort. Sending 5, 1281-byte ICMP Echos to 192.168.160.248, timeout is 2 seconds: Packet sent with a source address of 192.168.225.1=20 Packet sent with the DF bit set ..... Success rate is 0 percent (0/5) tcpdump on gif0 (large mtu before small mtu gif) [root@fw2-mru ~]# tcpdump -i gif0 -vvv -n host 192.168.225.1=20 tcpdump: listening on gif0, link-type NULL (BSD loopback), capture size 96 = bytes 17:55:54.006210 IP (tos 0x0, ttl 254, id 805, offset 0, flags [DF], proto I= CMP (1), length 1281) 192.168.225.1 > 192.168.160.248: ICMP echo request, id 161, seq 0, leng= th 1261 17:55:56.013039 IP (tos 0x0, ttl 254, id 806, offset 0, flags [DF], proto I= CMP (1), length 1281) 192.168.225.1 > 192.168.160.248: ICMP echo request, id 161, seq 1, leng= th 1261 17:55:58.015870 IP (tos 0x0, ttl 254, id 807, offset 0, flags [DF], proto I= CMP (1), length 1281) 192.168.225.1 > 192.168.160.248: ICMP echo request, id 161, seq 2, leng= th 1261 17:56:00.020833 IP (tos 0x0, ttl 254, id 808, offset 0, flags [DF], proto I= CMP (1), length 1281) 192.168.225.1 > 192.168.160.248: ICMP echo request, id 161, seq 3, leng= th 1261 17:56:02.027756 IP (tos 0x0, ttl 254, id 809, offset 0, flags [DF], proto I= CMP (1), length 1281) 192.168.225.1 > 192.168.160.248: ICMP echo request, id 161, seq 4, leng= th 1261 ^C 5 packets captured 99753 packets received by filter 0 packets dropped by kernel tcpdump on gif1 (small mtu on route to destination) (nothing) but if i omit df on cisco: [root@fw2-mru ~]# tcpdump -i gif1 -vvv -n host 192.168.225.1=20 tcpdump: listening on gif1, link-type NULL (BSD loopback), capture size 96 = bytes 17:59:03.083053 IP (tos 0x0, ttl 253, id 815, offset 0, flags [+], proto IC= MP (1), length 1276) 192.168.225.1 > 192.168.160.248: ICMP echo request, id 163, seq 0, leng= th 1256 17:59:03.083147 IP (tos 0x0, ttl 253, id 815, offset 1256, flags [none], pr= oto ICMP (1), length 25) 192.168.225.1 > 192.168.160.248: icmp 17:59:03.090882 IP (tos 0x0, ttl 253, id 816, offset 0, flags [+], proto IC= MP (1), length 1276) 192.168.225.1 > 192.168.160.248: ICMP echo request, id 163, seq 1, leng= th 1256 17:59:03.090976 IP (tos 0x0, ttl 253, id 816, offset 1256, flags [none], pr= oto ICMP (1), length 25) 192.168.225.1 > 192.168.160.248: icmp 17:59:03.097254 IP (tos 0x0, ttl 253, id 817, offset 0, flags [+], proto IC= MP (1), length 1276) 192.168.225.1 > 192.168.160.248: ICMP echo request, id 163, seq 2, leng= th 1256 17:59:03.097346 IP (tos 0x0, ttl 253, id 817, offset 1256, flags [none], pr= oto ICMP (1), length 25) 192.168.225.1 > 192.168.160.248: icmp 17:59:03.105749 IP (tos 0x0, ttl 253, id 818, offset 0, flags [+], proto IC= MP (1), length 1276) 192.168.225.1 > 192.168.160.248: ICMP echo request, id 163, seq 3, leng= th 1256 17:59:03.105844 IP (tos 0x0, ttl 253, id 818, offset 1256, flags [none], pr= oto ICMP (1), length 25) 192.168.225.1 > 192.168.160.248: icmp 17:59:03.115617 IP (tos 0x0, ttl 253, id 819, offset 0, flags [+], proto IC= MP (1), length 1276) 192.168.225.1 > 192.168.160.248: ICMP echo request, id 163, seq 4, leng= th 1256 17:59:03.115707 IP (tos 0x0, ttl 253, id 819, offset 1256, flags [none], pr= oto ICMP (1), length 25) 192.168.225.1 > 192.168.160.248: icmp e.g. destination reachable, fragmentation work, routes symmetrical. any comments ? --=20 =D1 =F3=E2=E0=E6=E5=ED=E8=E5=EC, Vladimir mailto:vl.varlog@gmail.com From owner-freebsd-net@FreeBSD.ORG Thu Sep 16 15:00:24 2010 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8AE27106573F for ; Thu, 16 Sep 2010 15:00:24 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 42C148FC2A for ; Thu, 16 Sep 2010 15:00:11 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o8GF0BT3077134 for ; Thu, 16 Sep 2010 15:00:11 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o8GF0AOC077109; Thu, 16 Sep 2010 15:00:10 GMT (envelope-from gnats) Date: Thu, 16 Sep 2010 15:00:10 GMT Message-Id: <201009161500.o8GF0AOC077109@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: dfilter@FreeBSD.ORG (dfilter service) Cc: Subject: Re: kern/146539: commit references a PR X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: dfilter service List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 15:00:24 -0000 The following reply was made to PR kern/146539; it has been noted by GNATS. From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/146539: commit references a PR Date: Thu, 16 Sep 2010 14:55:27 +0000 (UTC) Author: glebius Date: Thu Sep 16 14:55:22 2010 New Revision: 212735 URL: http://svn.freebsd.org/changeset/base/212735 Log: MFhead 210529: When installing a new ARP entry via 'arp -S', lla_lookup() will either find an existing entry, or allocate a new one. In the latter case an entry would have flags, that were supplied as argument to lla_lookup(). In case of an existing entry, flags aren't modified. This lead to losing LLE_PUB and/or LLE_PROXY flags. We should apply these flags either in lla_rt_output() or in the in.c:in_lltable_lookup(). It seems to me that lla_rt_output() is a more correct choice. PR: kern/148784, kern/146539 Modified: stable/8/sys/net/if_llatbl.c Directory Properties: stable/8/sys/ (props changed) stable/8/sys/amd64/include/xen/ (props changed) stable/8/sys/cddl/contrib/opensolaris/ (props changed) stable/8/sys/contrib/dev/acpica/ (props changed) stable/8/sys/contrib/pf/ (props changed) stable/8/sys/dev/xen/xenpci/ (props changed) Modified: stable/8/sys/net/if_llatbl.c ============================================================================== --- stable/8/sys/net/if_llatbl.c Thu Sep 16 14:30:32 2010 (r212734) +++ stable/8/sys/net/if_llatbl.c Thu Sep 16 14:55:22 2010 (r212735) @@ -337,6 +337,7 @@ lla_rt_output(struct rt_msghdr *rtm, str * LLE_DELETED flag, and reset the expiration timer */ bcopy(LLADDR(dl), &lle->ll_addr, ifp->if_addrlen); + lle->la_flags |= (flags & (LLE_PUB | LLE_PROXY)); lle->la_flags |= LLE_VALID; lle->la_flags &= ~LLE_DELETED; #ifdef INET6 _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Sep 16 15:08:14 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 71A471065672 for ; Thu, 16 Sep 2010 15:08:14 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 2B1538FC0A for ; Thu, 16 Sep 2010 15:08:14 +0000 (UTC) Received: by gwb15 with SMTP id 15so527153gwb.13 for ; Thu, 16 Sep 2010 08:08:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=hCFXoEhb+d8vmr//Vc3+stEvS3uylgcocnPqKOTYx1A=; b=pzLcTev4ohEifte/t12ikoh1jkrgI366zzTD61l0/VhQP4MqkcT8/mu2TIGDtpJSiH TJLoWpgMGAW8s/LxzXFP3+G69uKuw04YD2T3eS7cLE8J8b8ERQYQI3e8peZpmEA6SoCd qy0HqV9G8GK7F9q4vVU5R/rmCnuniAFYk25hg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=CjXWZvQqYXsg3znClHeQ8aJFMJ0seQqNS7xdSbOVVebjR8Wf/7wTbW8i58DvzWx6UL +dUjTOn9AEbG8G+l8B0aCSxLdPiblsgB1rvT4ARB4YFRxkMseVrxRwUAQZ4ZS95QDl0z dDC3RCw6cC7w0mT4/fOtExxW5033Jkfs5Gf58= MIME-Version: 1.0 Received: by 10.150.11.10 with SMTP id 10mr436267ybk.177.1284649693595; Thu, 16 Sep 2010 08:08:13 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.231.156.206 with HTTP; Thu, 16 Sep 2010 08:08:13 -0700 (PDT) In-Reply-To: <201009070131.42522.milu@dat.pl> References: <201009070131.42522.milu@dat.pl> Date: Thu, 16 Sep 2010 23:08:13 +0800 X-Google-Sender-Auth: 5E1TgWjwvsEYE9EltOgyFxgMRog Message-ID: From: Adrian Chadd To: Maciej Milewski Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org Subject: Re: ath wpa_supplicant timeouts on AR5416 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 15:08:14 -0000 On 7 September 2010 07:31, Maciej Milewski wrote: > The wpa_supplicant.conf isn't complicated: > network=3D{ > =A0 =A0 =A0 =A0ssid=3D"NET5" > =A0 =A0 =A0 =A0psk=3Dthelongpskphrase > } > > AFAIR this card was working fine in hostap mode. > > How can I help in fixing the issue? How long is the PSK? :) I'm not at all familiar with the crypto/keycache code in the atheros drivers yet, sorry. Have you tried a (much) shorter PSK? I've tested WPA/WEP in client mode on the AR5416; it worked for me. But then my PSKs are shorter (=3D< 9 characters) which may have an impact. Adrian From owner-freebsd-net@FreeBSD.ORG Thu Sep 16 15:08:55 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 51A0710656A3 for ; Thu, 16 Sep 2010 15:08:55 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: from eu1sys200aog108.obsmtp.com (eu1sys200aog108.obsmtp.com [207.126.144.125]) by mx1.freebsd.org (Postfix) with SMTP id CC8E38FC0A for ; Thu, 16 Sep 2010 15:08:52 +0000 (UTC) Received: from source ([63.174.175.251]) by eu1sys200aob108.postini.com ([207.126.147.11]) with SMTP ID DSNKTJIzA9gFhsTpyyqmjf2nmaiC6fRdxXWm@postini.com; Thu, 16 Sep 2010 15:08:53 UTC Received: from [172.17.10.53] (unknown [172.17.10.53]) by bbbx3.usdmm.com (Postfix) with ESMTP id D00CFFD01D; Thu, 16 Sep 2010 15:08:50 +0000 (UTC) Message-ID: <4C9232F9.70206@tomjudge.com> Date: Thu, 16 Sep 2010 10:08:41 -0500 From: Tom Judge User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.12) Gecko/20100826 Lightning/1.0b1 Thunderbird/3.0.7 MIME-Version: 1.0 To: freebsd-net@freebsd.org, bz@freebsd.org References: <273436110.20100916180053@gmail.com> In-Reply-To: <273436110.20100916180053@gmail.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: 7bit Cc: Subject: Re: Strange FreeBSD behavior when trying to forward beetween ipsec crypted gif's. May be a problem with ICMP unreach packets at all X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 15:08:55 -0000 On 09/16/2010 09:00 AM, Vladimir Grigorov wrote: > Greetings all. > > > I have strange problems related to passage icmp need-frag packets, and, as result, all packets with packets length greater than output gif MTU. > > Network diagram: > > [HostA] -- (mtu 1500) --- [FW1] --- ipsec gif mtu 1280 <-gif1 -- [FW2] - gif0 -> ipsec gif mtu 6100 - [FW3] -(mtu 1500) - [HostB] > > All FW's - Freebsd hosts > HostA - freebsd host > HostB - Cisco 3750e switch in L3 mode > > HostA can reach HostB and vice versa. Ping with length above 1280 works fine (pmtu = 1280). Ping with len=1281 without df bit also work fine. But ping with mtu 1281 fails. > > Question: Why FW2 does not send ICMP need-fragment-but-DF-set message to HostB ? > If you take a look at icmp_error() in sys/netinet/ip_icmp.c you will see that icmp errors are not sent for packets that have been previously been decrypted by IPSec. I have a feeling that this is because icmp_reflect() will not use the correct output path but I may be and most probably am wrong. Bjoern (CC'd) can probably shed more light on it. I am willing to spend some time on the fix for this if someone could give me a gentle nudge in the right direction. > I try to permit icmp from all interfaces on FW2, explicit send unreachable packet for all ip packets from defined source ip - nothing happens. I see increased packets counts related my source ip, but cant see responce icmps with unreachable code > > uname -a > FreeBSD fw2-mru.astrum-nival.com 8.0-RELEASE-p3 FreeBSD 8.0-RELEASE-p3 #3: Thu Jul 1 18:24:35 MSD 2010 root@fw2-mru.astrum-nival.com:/usr/obj/usr/src/sys/gw amd64 > > ifconfig gif0 > gif0: flags=8051 metric 0 mtu 6100 > tunnel inet 217.69.143.28 --> 217.69.143.57 > inet 10.192.224.5 --> 10.192.224.6 netmask 0xfffffffc > options=1 > > ifconfig gif1 > gif1: flags=8051 metric 0 mtu 1280 > tunnel inet 217.69.143.28 --> 88.212.205.166 > inet 10.160.192.6 --> 10.160.192.5 netmask 0xfffffffc > options=1 > > netstat -nr | grep 192.168.224 > > 192.168.224.0/22 10.192.224.6 UG1 0 36031303 gif0 > > netstat -nr | grep 192.168.160. > 192.168.160.0/24 10.160.192.5 UG1 0 10969867 gif1 > > # ipfw show > 00006 10 6505 allow icmp from any to 192.168.225.1 via gif0 > 00100 10524445 1225052712 allow ip from any to any via lo0 > 00200 0 0 deny ip from any to 127.0.0.0/8 > 00300 0 0 deny ip from 127.0.0.0/8 to any > 00305 2054 433651 allow icmp from any to any via gif0 icmptypes 3,11 > 00306 0 0 allow icmp from any to 192.168.225.1 via gif0 > 00310 6960 575159 nat 220 ip from table(10) to any via vlan220 > 00315 1198 70832 deny ip from not me to 192.168.66.0/23 out xmit vlan220 > 00320 6512 1611912 nat 220 ip from 192.168.66.0/23 to 192.168.13.199 in recv vlan220 > 00400 114560294 8963623578 nat 123 ip from 192.168.196.0/24 to any out via vlan506 > 00402 36831424 2199804860 nat 123 ip from 192.168.193.0/24 to any out via vlan506 > 00403 153380 9265905 nat 123 ip from 192.168.197.0/24 to any out via vlan506 > 00500 0 0 nat 123 ip from any to 195.211.130.9 in via vlan506 > 00501 147593882 174870597871 nat 123 ip from any to 195.211.130.9 in via vlan500 > 01100 0 0 allow tcp from table(21) to table(23) dst-port 29000 > 01110 0 0 deny tcp from table(22) to table(23) dst-port 29000 > 01120 3 144 deny tcp from table(20) to table(23) dst-port 29000 > 65530 589120438508 133855063718386 allow ip from any to any > 65535 0 0 deny ip from any to any > > try to ping from cisco: > > c3750e.gldn#ping 192.168.160.248 source 192.168.225.1 repea 5 size 1281 df > > Type escape sequence to abort. > Sending 5, 1281-byte ICMP Echos to 192.168.160.248, timeout is 2 seconds: > Packet sent with a source address of 192.168.225.1 > Packet sent with the DF bit set > ..... > Success rate is 0 percent (0/5) > > tcpdump on gif0 (large mtu before small mtu gif) > > [root@fw2-mru ~]# tcpdump -i gif0 -vvv -n host 192.168.225.1 > tcpdump: listening on gif0, link-type NULL (BSD loopback), capture size 96 bytes > 17:55:54.006210 IP (tos 0x0, ttl 254, id 805, offset 0, flags [DF], proto ICMP (1), length 1281) > 192.168.225.1 > 192.168.160.248: ICMP echo request, id 161, seq 0, length 1261 > 17:55:56.013039 IP (tos 0x0, ttl 254, id 806, offset 0, flags [DF], proto ICMP (1), length 1281) > 192.168.225.1 > 192.168.160.248: ICMP echo request, id 161, seq 1, length 1261 > 17:55:58.015870 IP (tos 0x0, ttl 254, id 807, offset 0, flags [DF], proto ICMP (1), length 1281) > 192.168.225.1 > 192.168.160.248: ICMP echo request, id 161, seq 2, length 1261 > 17:56:00.020833 IP (tos 0x0, ttl 254, id 808, offset 0, flags [DF], proto ICMP (1), length 1281) > 192.168.225.1 > 192.168.160.248: ICMP echo request, id 161, seq 3, length 1261 > 17:56:02.027756 IP (tos 0x0, ttl 254, id 809, offset 0, flags [DF], proto ICMP (1), length 1281) > 192.168.225.1 > 192.168.160.248: ICMP echo request, id 161, seq 4, length 1261 > ^C > 5 packets captured > 99753 packets received by filter > 0 packets dropped by kernel > > tcpdump on gif1 (small mtu on route to destination) > > (nothing) > > but if i omit df on cisco: > > [root@fw2-mru ~]# tcpdump -i gif1 -vvv -n host 192.168.225.1 > tcpdump: listening on gif1, link-type NULL (BSD loopback), capture size 96 bytes > 17:59:03.083053 IP (tos 0x0, ttl 253, id 815, offset 0, flags [+], proto ICMP (1), length 1276) > 192.168.225.1 > 192.168.160.248: ICMP echo request, id 163, seq 0, length 1256 > 17:59:03.083147 IP (tos 0x0, ttl 253, id 815, offset 1256, flags [none], proto ICMP (1), length 25) > 192.168.225.1 > 192.168.160.248: icmp > 17:59:03.090882 IP (tos 0x0, ttl 253, id 816, offset 0, flags [+], proto ICMP (1), length 1276) > 192.168.225.1 > 192.168.160.248: ICMP echo request, id 163, seq 1, length 1256 > 17:59:03.090976 IP (tos 0x0, ttl 253, id 816, offset 1256, flags [none], proto ICMP (1), length 25) > 192.168.225.1 > 192.168.160.248: icmp > 17:59:03.097254 IP (tos 0x0, ttl 253, id 817, offset 0, flags [+], proto ICMP (1), length 1276) > 192.168.225.1 > 192.168.160.248: ICMP echo request, id 163, seq 2, length 1256 > 17:59:03.097346 IP (tos 0x0, ttl 253, id 817, offset 1256, flags [none], proto ICMP (1), length 25) > 192.168.225.1 > 192.168.160.248: icmp > 17:59:03.105749 IP (tos 0x0, ttl 253, id 818, offset 0, flags [+], proto ICMP (1), length 1276) > 192.168.225.1 > 192.168.160.248: ICMP echo request, id 163, seq 3, length 1256 > 17:59:03.105844 IP (tos 0x0, ttl 253, id 818, offset 1256, flags [none], proto ICMP (1), length 25) > 192.168.225.1 > 192.168.160.248: icmp > 17:59:03.115617 IP (tos 0x0, ttl 253, id 819, offset 0, flags [+], proto ICMP (1), length 1276) > 192.168.225.1 > 192.168.160.248: ICMP echo request, id 163, seq 4, length 1256 > 17:59:03.115707 IP (tos 0x0, ttl 253, id 819, offset 1256, flags [none], proto ICMP (1), length 25) > 192.168.225.1 > 192.168.160.248: icmp > > e.g. destination reachable, fragmentation work, routes symmetrical. > > any comments ? > > > Good Luck Tom -- TJU13-ARIN From owner-freebsd-net@FreeBSD.ORG Thu Sep 16 15:58:50 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DBAAE1065675 for ; Thu, 16 Sep 2010 15:58:50 +0000 (UTC) (envelope-from milu@dat.pl) Received: from jab.dat.pl (dat.pl [80.51.155.34]) by mx1.freebsd.org (Postfix) with ESMTP id 40E058FC21 for ; Thu, 16 Sep 2010 15:58:49 +0000 (UTC) Received: from jab.dat.pl (jsrv.dat.pl [127.0.0.1]) by jab.dat.pl (Postfix) with ESMTP id 089C951; Thu, 16 Sep 2010 17:58:48 +0200 (CEST) X-Virus-Scanned: amavisd-new at dat.pl Received: from jab.dat.pl ([127.0.0.1]) by jab.dat.pl (jab.dat.pl [127.0.0.1]) (amavisd-new, port 10024) with LMTP id XGM3-oaxN7AY; Thu, 16 Sep 2010 17:58:44 +0200 (CEST) Received: from snifi.localnet (unknown [212.69.68.42]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by jab.dat.pl (Postfix) with ESMTPSA id 568BE2D; Thu, 16 Sep 2010 17:58:44 +0200 (CEST) From: Maciej Milewski To: Adrian Chadd Date: Thu, 16 Sep 2010 17:58:44 +0200 User-Agent: KMail/1.13.5 (Linux/2.6.35-ARCH; KDE/4.5.1; x86_64; ; ) References: <201009070131.42522.milu@dat.pl> In-Reply-To: X-KMail-Markup: true MIME-Version: 1.0 Message-Id: <201009161758.45734.milu@dat.pl> Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org Subject: Re: ath wpa_supplicant timeouts on AR5416 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 15:58:50 -0000 Dnia czwartek 16 wrzesie=F1 2010 o 17:08:13 Adrian Chadd napisa=B3(a): > On 7 September 2010 07:31, Maciej Milewski wrote: > > The wpa_supplicant.conf isn't complicated: > > network=3D{ > > ssid=3D"NET5" > > psk=3Dthelongpskphrase > > } > >=20 > > AFAIR this card was working fine in hostap mode. > >=20 > > How can I help in fixing the issue? >=20 > How long is the PSK? :) 10 chars long without any special chars :) >=20 > I'm not at all familiar with the crypto/keycache code in the atheros > drivers yet, sorry. Have you tried a (much) shorter PSK? Well, I thought that 10 chars is short enough for the test but I can try ot= her=20 PSK. >=20 > I've tested WPA/WEP in client mode on the AR5416; it worked for me. > But then my PSKs are shorter (=3D< 9 characters) which may have an > impact. >=20 >=20 > Adrian I can try even smaller PSK but how to explain that on other ath based cards= I=20 have no problem connecting to the network? Maciek From owner-freebsd-net@FreeBSD.ORG Thu Sep 16 19:38:57 2010 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9A265106564A for ; Thu, 16 Sep 2010 19:38:57 +0000 (UTC) (envelope-from if@xip.at) Received: from chile.gbit.at (ns1.xip.at [193.239.188.99]) by mx1.freebsd.org (Postfix) with ESMTP id 606D08FC12 for ; Thu, 16 Sep 2010 19:38:55 +0000 (UTC) Received: (qmail 11673 invoked from network); 16 Sep 2010 21:38:53 +0200 Received: from unknown (HELO filebunker.xip.at) (86.59.10.180) by chile.gbit.at with (DHE-RSA-AES256-SHA encrypted) SMTP; 16 Sep 2010 21:38:53 +0200 Date: Thu, 16 Sep 2010 21:38:51 +0200 (CEST) From: Ingo Flaschberger To: Qing Li In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (LFD 1167 2008-08-23) MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="168430090-1874193526-1284665933=:5369" Cc: "Li, Qing" , net@freebsd.org Subject: Re: funny ECMP X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 19:38:57 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --168430090-1874193526-1284665933=:5369 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Hi Qing, version 4 o patch... fixed deleting interface loopback routes. Kind regards, Ingo Flaschberger --168430090-1874193526-1284665933=:5369 Content-Type: TEXT/plain; name=mpath_patch_if_v4.txt Content-Transfer-Encoding: BASE64 Content-ID: Content-Description: Content-Disposition: attachment; filename=mpath_patch_if_v4.txt ZGlmZiAtdSAtciAvdXNyX2RpZmYvc3JjL3N5cy9jb250cmliL2lwZmlsdGVy L25ldGluZXQvaXBfcG9vbC5jIC9yb3V0ZXIvdXNyL3NyYy9zeXMvY29udHJp Yi9pcGZpbHRlci9uZXRpbmV0L2lwX3Bvb2wuYw0KLS0tIC91c3JfZGlmZi9z cmMvc3lzL2NvbnRyaWIvaXBmaWx0ZXIvbmV0aW5ldC9pcF9wb29sLmMJMjAw Ny0xMC0xOCAyMTo0MjozOC4wMDAwMDAwMDAgKzAwMDANCisrKyAvcm91dGVy L3Vzci9zcmMvc3lzL2NvbnRyaWIvaXBmaWx0ZXIvbmV0aW5ldC9pcF9wb29s LmMJMjAxMC0wOS0xMSAwMTowMjozMS4wMDAwMDAwMDAgKzAwMDANCkBAIC02 MjAsNyArNjIwLDcgQEANCiANCiAJUkFESVhfTk9ERV9IRUFEX0xPQ0soaXBv LT5pcG9faGVhZCk7DQogCWlwby0+aXBvX2hlYWQtPnJuaF9kZWxhZGRyKCZp cGUtPmlwbl9hZGRyLCAmaXBlLT5pcG5fbWFzaywNCi0JCQkJICAgaXBvLT5p cG9faGVhZCk7DQorCQkJCSAgIGlwby0+aXBvX2hlYWQsIE5VTEwpOw0KIAlS QURJWF9OT0RFX0hFQURfVU5MT0NLKGlwby0+aXBvX2hlYWQpOw0KIA0KIAlp cF9wb29sX25vZGVfZGVyZWYoaXBlKTsNCkBAIC03NTEsNyArNzUxLDcgQEAN CiAJUkFESVhfTk9ERV9IRUFEX0xPQ0soaXBvLT5pcG9faGVhZCk7DQogCXdo aWxlICgobiA9IGlwby0+aXBvX2xpc3QpICE9IE5VTEwpIHsNCiAJCWlwby0+ aXBvX2hlYWQtPnJuaF9kZWxhZGRyKCZuLT5pcG5fYWRkciwgJm4tPmlwbl9t YXNrLA0KLQkJCQkJICAgaXBvLT5pcG9faGVhZCk7DQorCQkJCQkgICBpcG8t Pmlwb19oZWFkLCBOVUxMKTsNCiANCiAJCSpuLT5pcG5fcG5leHQgPSBuLT5p cG5fbmV4dDsNCiAJCWlmIChuLT5pcG5fbmV4dCkNCkBAIC05NjMsNyArOTYz LDcgQEANCiAJc3RydWN0IHJhZGl4X25vZGVfaGVhZCAqcm5oID0gcDsNCiAJ c3RydWN0IHJhZGl4X25vZGUgKmQ7DQogDQotCWQgPSBybmgtPnJuaF9kZWxh ZGRyKG4tPnJuX2tleSwgTlVMTCwgcm5oKTsNCisJZCA9IHJuaC0+cm5oX2Rl bGFkZHIobi0+cm5fa2V5LCBOVUxMLCBybmgsIE5VTEwpOw0KIAlpZiAoZCAh PSBOVUxMKSB7DQogCQlGcmVlUyhkLCBtYXhfa2V5bGVuICsgMiAqIHNpemVv ZiAoKmQpKTsNCiAJfQ0KZGlmZiAtdSAtciAvdXNyX2RpZmYvc3JjL3N5cy9j b250cmliL3BmL25ldC9wZi5jIC9yb3V0ZXIvdXNyL3NyYy9zeXMvY29udHJp Yi9wZi9uZXQvcGYuYw0KLS0tIC91c3JfZGlmZi9zcmMvc3lzL2NvbnRyaWIv cGYvbmV0L3BmLmMJMjAxMC0wMS0yMyAwMDozMjoxOS4wMDAwMDAwMDAgKzAw MDANCisrKyAvcm91dGVyL3Vzci9zcmMvc3lzL2NvbnRyaWIvcGYvbmV0L3Bm LmMJMjAxMC0wOC0yNiAxNToxMjoxMi4wMDAwMDAwMDAgKzAwMDANCkBAIC05 OSw5ICs5OSw3IEBADQogI2luY2x1ZGUgPG5ldC9pZl90eXBlcy5oPg0KICNp bmNsdWRlIDxuZXQvYnBmLmg+DQogI2luY2x1ZGUgPG5ldC9yb3V0ZS5oPg0K LSNpZm5kZWYgX19GcmVlQlNEX18NCiAjaW5jbHVkZSA8bmV0L3JhZGl4X21w YXRoLmg+DQotI2VuZGlmDQogDQogI2luY2x1ZGUgPG5ldGluZXQvaW4uaD4N CiAjaW5jbHVkZSA8bmV0aW5ldC9pbl92YXIuaD4NCkBAIC02MTY2LDkgKzYx NjQsOSBAQA0KIAkJCWlmIChraWYtPnBmaWtfaWZwID09IGlmcCkNCiAJCQkJ cmV0ID0gMTsNCiAjaWZkZWYgX19GcmVlQlNEX18gLyogTVVMVElQQVRIX1JP VVRJTkcgKi8NCi0JCQlybiA9IE5VTEw7DQorCQkJcm4gPSBybl9tcGF0aF9u ZXh0KHJuKTsgLyogWFhYIHdhcyBiZWZvcmU6IHJuID0gTlVMTDsgKi8NCiAj ZWxzZQ0KLQkJCXJuID0gcm5fbXBhdGhfbmV4dChybik7DQorCQkJcm4gPSBy bl9tcGF0aF9uZXh0KHJuLCAwKTsNCiAjZW5kaWYNCiAJCX0gd2hpbGUgKGNo ZWNrX21wYXRoID09IDEgJiYgcm4gIT0gTlVMTCAmJiByZXQgPT0gMCk7DQog CX0gZWxzZQ0KZGlmZiAtdSAtciAvdXNyX2RpZmYvc3JjL3N5cy9jb250cmli L3BmL25ldC9wZl90YWJsZS5jIC9yb3V0ZXIvdXNyL3NyYy9zeXMvY29udHJp Yi9wZi9uZXQvcGZfdGFibGUuYw0KLS0tIC91c3JfZGlmZi9zcmMvc3lzL2Nv bnRyaWIvcGYvbmV0L3BmX3RhYmxlLmMJMjAwOS0wOC0wMyAwODoxMzowNi4w MDAwMDAwMDAgKzAwMDANCisrKyAvcm91dGVyL3Vzci9zcmMvc3lzL2NvbnRy aWIvcGYvbmV0L3BmX3RhYmxlLmMJMjAxMC0wOC0yNiAxMTo1MDozMS4wMDAw MDAwMDAgKzAwMDANCkBAIC0xMTE0LDE3ICsxMTE0LDkgQEANCiAjZW5kaWYN CiAJaWYgKEtFTlRSWV9ORVRXT1JLKGtlKSkgew0KIAkJcGZyX3ByZXBhcmVf bmV0d29yaygmbWFzaywga2UtPnBmcmtlX2FmLCBrZS0+cGZya2VfbmV0KTsN Ci0jaWZkZWYgX19GcmVlQlNEX18NCi0JCXJuID0gcm5fZGVsZXRlKCZrZS0+ cGZya2Vfc2EsICZtYXNrLCBoZWFkKTsNCi0jZWxzZQ0KIAkJcm4gPSBybl9k ZWxldGUoJmtlLT5wZnJrZV9zYSwgJm1hc2ssIGhlYWQsIE5VTEwpOw0KLSNl bmRpZg0KIAl9IGVsc2UNCi0jaWZkZWYgX19GcmVlQlNEX18NCi0JCXJuID0g cm5fZGVsZXRlKCZrZS0+cGZya2Vfc2EsIE5VTEwsIGhlYWQpOw0KLSNlbHNl DQogCQlybiA9IHJuX2RlbGV0ZSgma2UtPnBmcmtlX3NhLCBOVUxMLCBoZWFk LCBOVUxMKTsNCi0jZW5kaWYNCiAJc3BseChzKTsNCiANCiAJaWYgKHJuID09 IE5VTEwpIHsNCmRpZmYgLXUgLXIgL3Vzcl9kaWZmL3NyYy9zeXMvbmV0L3Jh ZGl4LmMgL3JvdXRlci91c3Ivc3JjL3N5cy9uZXQvcmFkaXguYw0KLS0tIC91 c3JfZGlmZi9zcmMvc3lzL25ldC9yYWRpeC5jCTIwMTAtMDQtMDIgMDU6MDI6 NTAuMDAwMDAwMDAwICswMDAwDQorKysgL3JvdXRlci91c3Ivc3JjL3N5cy9u ZXQvcmFkaXguYwkyMDEwLTA5LTExIDAxOjEzOjM0LjAwMDAwMDAwMCArMDAw MA0KQEAgLTYxNCw3ICs2MTQsNyBAQA0KIAlzdHJ1Y3QgcmFkaXhfbm9kZSB0 cmVlbm9kZXNbMl07DQogew0KIAljYWRkcl90IHYgPSAoY2FkZHJfdCl2X2Fy ZywgbmV0bWFzayA9IChjYWRkcl90KW5fYXJnOw0KLQlyZWdpc3RlciBzdHJ1 Y3QgcmFkaXhfbm9kZSAqdCwgKnggPSAwLCAqdHQ7DQorCXJlZ2lzdGVyIHN0 cnVjdCByYWRpeF9ub2RlICp0LCAqeCA9IDAsICp4eCA9IDAsICp0dDsNCiAJ c3RydWN0IHJhZGl4X25vZGUgKnNhdmVkX3R0LCAqdG9wID0gaGVhZC0+cm5o X3RyZWV0b3A7DQogCXNob3J0IGIgPSAwLCBiX2xlYWYgPSAwOw0KIAlpbnQg a2V5ZHVwbGljYXRlZDsNCkBAIC03MjMsMTIgKzcyMywxOSBAQA0KIAkJeCA9 IHQtPnJuX3JpZ2h0Ow0KIAkvKiBQcm9tb3RlIGdlbmVyYWwgcm91dGVzIGZy b20gYmVsb3cgKi8NCiAJaWYgKHgtPnJuX2JpdCA8IDApIHsNCi0JICAgIGZv ciAobXAgPSAmdC0+cm5fbWtsaXN0OyB4OyB4ID0geC0+cm5fZHVwZWRrZXkp DQorCSAgICBmb3IgKG1wID0gJnQtPnJuX21rbGlzdDsgeDsgeHggPSB4LCB4 ID0geC0+cm5fZHVwZWRrZXkpIHsNCisJCWlmICh4eCAmJiB4eC0+cm5fbWts aXN0ICYmIHh4LT5ybl9tYXNrID09IHgtPnJuX21hc2sgJiYNCisJCQkJeC0+ cm5fbWtsaXN0ID09IDApIHsNCisJCQkvKiBtdWx0aXBhdGggcm91dGUsIGJ1 bXAgcmVmY291bnQgb24gZmlyc3QgbWtsaXN0ICovDQorCQkJeC0+cm5fbWts aXN0ID0geHgtPnJuX21rbGlzdDsNCisJCQl4LT5ybl9ta2xpc3QtPnJtX3Jl ZnMrKzsNCisJCX0NCiAJCWlmICh4LT5ybl9tYXNrICYmICh4LT5ybl9iaXQg Pj0gYl9sZWFmKSAmJiB4LT5ybl9ta2xpc3QgPT0gMCkgew0KIAkJCSptcCA9 IG0gPSBybl9uZXdfcmFkaXhfbWFzayh4LCAwKTsNCiAJCQlpZiAobSkNCiAJ CQkJbXAgPSAmbS0+cm1fbWtsaXN0Ow0KIAkJfQ0KKwkgICAgfQ0KIAl9IGVs c2UgaWYgKHgtPnJuX21rbGlzdCkgew0KIAkJLyoNCiAJCSAqIFNraXAgb3Zl ciBtYXNrcyB3aG9zZSBpbmRleCBpcyA+IHRoYXQgb2YgbmV3IG5vZGUNCkBA IC03NjAsMTEgKzc2NywzMCBAQA0KIAkJCWJyZWFrOw0KIAkJaWYgKG0tPnJt X2ZsYWdzICYgUk5GX05PUk1BTCkgew0KIAkJCW1tYXNrID0gbS0+cm1fbGVh Zi0+cm5fbWFzazsNCi0JCQlpZiAodHQtPnJuX2ZsYWdzICYgUk5GX05PUk1B TCkgew0KLSNpZiAhZGVmaW5lZChSQURJWF9NUEFUSCkNCi0JCQkgICAgbG9n KExPR19FUlIsDQotCQkJICAgICAgICAiTm9uLXVuaXF1ZSBub3JtYWwgcm91 dGUsIG1hc2sgbm90IGVudGVyZWRcbiIpOw0KKyAgICAgICAgICAgICAgICAg ICAgICAgIGlmIChrZXlkdXBsaWNhdGVkKSB7DQorCQkJCWlmIChtLT5ybV9s ZWFmLT5ybl9wYXJlbnQgPT0gdHQpDQorCQkJCQkvKiBuZXcgcm91dGUgaXMg YmV0dHRlciAqLw0KKwkJCQkJbS0+cm1fbGVhZiA9IHR0Ow0KKyNpZmRlZiBE SUFHTk9TVElDDQorCQkJCWVsc2Ugew0KKwkJCQkJZm9yICh0ID0gbS0+cm1f bGVhZjsgdDsNCisJCQkJCQl0ID0gdC0+cm5fZHVwZWRrZXkpIHsNCisJCQkJ CQkJYnJlYWs7DQorCQkJCQl9DQorCQkJCQlpZiAodCA9PSBOVUxMKSB7DQor CQkJCQkJbG9nKExPR19FUlIsICJOb24tdW5pcXVlICINCisJCQkJCQkJIm5v cm1hbCByb3V0ZSBvbiBkdXBlZGtleSwgIg0KKwkJCQkJCQkibWFzayBub3Qg ZW50ZXJlZFxuIik7DQorCQkJCQkJcmV0dXJuIHR0Ow0KKwkJCQkJfQ0KKwkJ CQl9DQogI2VuZGlmDQorCQkJCW0tPnJtX3JlZnMrKzsNCisJCQkJdHQtPnJu X21rbGlzdCA9IG07DQorCQkJCXJldHVybiB0dDsNCisJCQkgfSBlbHNlIGlm ICh0dC0+cm5fZmxhZ3MgJiBSTkZfTk9STUFMKSB7DQorCQkJCWxvZyhMT0df RVJSLCAiTm9uLXVuaXF1ZSBub3JtYWwgcm91dGUsIg0KKwkJCQkJIiBtYXNr IG5vdCBlbnRlcmVkXG4iKTsNCiAJCQkJcmV0dXJuIHR0Ow0KIAkJCX0NCiAJ CX0gZWxzZQ0KQEAgLTc4Myw5ICs4MDksMTAgQEANCiB9DQogDQogc3RydWN0 IHJhZGl4X25vZGUgKg0KLXJuX2RlbGV0ZSh2X2FyZywgbmV0bWFza19hcmcs IGhlYWQpDQorcm5fZGVsZXRlKHZfYXJnLCBuZXRtYXNrX2FyZywgaGVhZCwg cm4pDQogCXZvaWQgKnZfYXJnLCAqbmV0bWFza19hcmc7DQogCXN0cnVjdCBy YWRpeF9ub2RlX2hlYWQgKmhlYWQ7DQorCXN0cnVjdCByYWRpeF9ub2RlICpy bjsNCiB7DQogCXJlZ2lzdGVyIHN0cnVjdCByYWRpeF9ub2RlICp0LCAqcCwg KngsICp0dDsNCiAJc3RydWN0IHJhZGl4X21hc2sgKm0sICpzYXZlZF9tLCAq Km1wOw0KQEAgLTgxNSwxMyArODQyLDM4IEBADQogCQkJaWYgKCh0dCA9IHR0 LT5ybl9kdXBlZGtleSkgPT0gMCkNCiAJCQkJcmV0dXJuICgwKTsNCiAJfQ0K KyNpZmRlZiBSQURJWF9NUEFUSA0KKwlpZiAocm4pIHsNCisJCXdoaWxlICh0 dCAhPSBybikNCisJCQlpZiAoKHR0ID0gdHQtPnJuX2R1cGVka2V5KSA9PSAw KQ0KKwkJCQlyZXR1cm4gKDApOw0KKwl9DQorI2VuZGlmDQorCQ0KIAlpZiAo dHQtPnJuX21hc2sgPT0gMCB8fCAoc2F2ZWRfbSA9IG0gPSB0dC0+cm5fbWts aXN0KSA9PSAwKQ0KIAkJZ290byBvbjE7DQogCWlmICh0dC0+cm5fZmxhZ3Mg JiBSTkZfTk9STUFMKSB7DQotCQlpZiAobS0+cm1fbGVhZiAhPSB0dCB8fCBt LT5ybV9yZWZzID4gMCkgew0KKwkJaWYgKG0tPnJtX2xlYWYgIT0gdHQgJiYg bS0+cm1fcmVmcyA9PSAwKSB7DQogCQkJbG9nKExPR19FUlIsICJybl9kZWxl dGU6IGluY29uc2lzdGVudCBhbm5vdGF0aW9uXG4iKTsNCiAJCQlyZXR1cm4g MDsgIC8qIGRhbmdsaW5nIHJlZiBjb3VsZCBjYXVzZSBkaXNhc3RlciAqLw0K IAkJfQ0KKwkJaWYgKG0tPnJtX2xlYWYgIT0gdHQpIHsNCisJCQlpZiAoLS1t LT5ybV9yZWZzID49IDApDQorCQkJCWdvdG8gb24xOw0KKyAgICAgICAgICAg ICAgICB9DQorCQkvKiB0dCBpcyBjdXJyZW50bHkgdGhlIGhlYWQgb2YgdGhl IHBvc3NpYmxlIG11bHRpcGF0aCBjaGFpbiAqLw0KKwkJaWYgKG0tPnJtX3Jl ZnMgPiAwKSB7DQorCQkJaWYgKHR0LT5ybl9kdXBlZGtleSA9PSBOVUxMIHx8 DQorCQkJICAgIHR0LT5ybl9kdXBlZGtleS0+cm5fbWtsaXN0ICE9IG0pIHsN CisJCQkJbG9nKExPR19FUlIsICJybl9kZWxldGU6IGluY29uc2lzdGVudCAi DQorCQkJCSAgICAiZHVwZWRrZXkgbGlzdFxuIik7DQorCQkJCXJldHVybiAo MCk7DQorCQkJfQ0KKwkJCW0tPnJtX2xlYWYgPSB0dC0+cm5fZHVwZWRrZXk7 DQorCQkJLS1tLT5ybV9yZWZzOw0KKwkJCWdvdG8gb24xOw0KKwkJfQ0KKwkJ LyogZWxzZSB0dCBpcyBsYXN0IGFuZCBvbmx5IHJvdXRlICovDQogCX0gZWxz ZSB7DQogCQlpZiAobS0+cm1fbWFzayAhPSB0dC0+cm5fbWFzaykgew0KIAkJ CWxvZyhMT0dfRVJSLCAicm5fZGVsZXRlOiBpbmNvbnNpc3RlbnQgYW5ub3Rh dGlvblxuIik7DQpAQCAtODY5LDIxICs5MjEsMTcgQEANCiAJCSAqLw0KIAkJ aWYgKHR0ID09IHNhdmVkX3R0KSB7DQogCQkJLyogcmVtb3ZlIGZyb20gaGVh ZCBvZiBjaGFpbiAqLw0KLQkJCXggPSBkdXBlZGtleTsgeC0+cm5fcGFyZW50 ID0gdDsNCisJCQl4ID0gZHVwZWRrZXk7IA0KKwkJCXgtPnJuX3BhcmVudCA9 IHQ7DQogCQkJaWYgKHQtPnJuX2xlZnQgPT0gdHQpDQogCQkJCXQtPnJuX2xl ZnQgPSB4Ow0KIAkJCWVsc2UNCiAJCQkJdC0+cm5fcmlnaHQgPSB4Ow0KIAkJ fSBlbHNlIHsNCi0JCQkvKiBmaW5kIG5vZGUgaW4gZnJvbnQgb2YgdHQgb24g dGhlIGNoYWluICovDQotCQkJZm9yICh4ID0gcCA9IHNhdmVkX3R0OyBwICYm IHAtPnJuX2R1cGVka2V5ICE9IHR0OykNCi0JCQkJcCA9IHAtPnJuX2R1cGVk a2V5Ow0KLQkJCWlmIChwKSB7DQotCQkJCXAtPnJuX2R1cGVka2V5ID0gdHQt PnJuX2R1cGVka2V5Ow0KLQkJCQlpZiAodHQtPnJuX2R1cGVka2V5KQkJLyog cGFyZW50ICovDQotCQkJCQl0dC0+cm5fZHVwZWRrZXktPnJuX3BhcmVudCA9 IHA7DQotCQkJCQkJCQkvKiBwYXJlbnQgKi8NCi0JCQl9IGVsc2UgbG9nKExP R19FUlIsICJybl9kZWxldGU6IGNvdWxkbid0IGZpbmQgdXNcbiIpOw0KKwkJ CXggPSBzYXZlZF90dDsNCisJCQl0LT5ybl9kdXBlZGtleSA9IHR0LT5ybl9k dXBlZGtleTsNCisJCQlpZiAodHQtPnJuX2R1cGVka2V5KQ0KKwkJCQl0dC0+ cm5fZHVwZWRrZXktPnJuX3BhcmVudCA9IHQ7DQogCQl9DQogCQl0ID0gdHQg KyAxOw0KIAkJaWYgICh0LT5ybl9mbGFncyAmIFJORl9BQ1RJVkUpIHsNCkBA IC05MzEsMTQgKzk3OSwyMSBAQA0KIAkJCQlpZiAobSA9PSB4LT5ybl9ta2xp c3QpIHsNCiAJCQkJCXN0cnVjdCByYWRpeF9tYXNrICptbSA9IG0tPnJtX21r bGlzdDsNCiAJCQkJCXgtPnJuX21rbGlzdCA9IDA7DQotCQkJCQlpZiAoLS0o bS0+cm1fcmVmcykgPCAwKQ0KKwkJCQkJaWYgKC0tKG0tPnJtX3JlZnMpIDwg MCkgew0KIAkJCQkJCU1LRnJlZShtKTsNCisJCQkJCX0gZWxzZSBpZiAobS0+ cm1fZmxhZ3MgJiBSTkZfTk9STUFMKSB7DQorCQkJCQkJLyoNCisJCQkJCQkg KiBkb24ndCBwcm9ncmVzcyBiZWNhdXNlIHRoaXMNCisJCQkJCQkgKiBhIG11 bHRpcGF0aCByb3V0ZS4gTmV4dA0KKwkJCQkJCSAqIHJvdXRlIHdpbGwgdXNl IHRoZSBzYW1lIG0uDQorCQkJCQkJICovDQorCQkJCQkJbW0gPSBtOw0KKwkJ CQkJfQ0KIAkJCQkJbSA9IG1tOw0KIAkJCQl9DQogCQkJaWYgKG0pDQogCQkJ CWxvZyhMT0dfRVJSLA0KLQkJCQkgICAgInJuX2RlbGV0ZTogT3JwaGFuZWQg TWFzayAlcCBhdCAlcFxuIiwNCi0JCQkJICAgIG0sIHgpOw0KKwkJCQkgICAg InJuX2RlbGV0ZTogT3JwaGFuZWQgTWFzayAlcCBhdCAlcFxuIiwgbSwgeCk7 DQogCQl9DQogCX0NCiAJLyoNCkBAIC05OTAsMTEgKzEwNDUsOCBAQA0KIAkg KiBybl9zZWFyY2hfbSBpcyBzb3J0LW9mLW9wZW4tY29kZWQgaGVyZS4gV2Ug Y2Fubm90IHVzZSB0aGUNCiAJICogZnVuY3Rpb24gYmVjYXVzZSB3ZSBuZWVk IHRvIGtlZXAgdHJhY2sgb2YgdGhlIGxhc3Qgbm9kZSBzZWVuLg0KIAkgKi8N Ci0JLyogcHJpbnRmKCJhYm91dCB0byBzZWFyY2hcbiIpOyAqLw0KIAlmb3Ig KHJuID0gaC0+cm5oX3RyZWV0b3A7IHJuLT5ybl9iaXQgPj0gMDsgKSB7DQog CQlsYXN0ID0gcm47DQotCQkvKiBwcmludGYoInJuX2JpdCAlZCwgcm5fYm1h c2sgJXgsIHhtW3JuX29mZnNldF0gJXhcbiIsDQotCQkgICAgICAgcm4tPnJu X2JpdCwgcm4tPnJuX2JtYXNrLCB4bVtybi0+cm5fb2Zmc2V0XSk7ICovDQog CQlpZiAoIShybi0+cm5fYm1hc2sgJiB4bVtybi0+cm5fb2Zmc2V0XSkpIHsN CiAJCQlicmVhazsNCiAJCX0NCkBAIC0xMDA0LDcgKzEwNTYsNiBAQA0KIAkJ CXJuID0gcm4tPnJuX2xlZnQ7DQogCQl9DQogCX0NCi0JLyogcHJpbnRmKCJk b25lIHNlYXJjaGluZ1xuIik7ICovDQogDQogCS8qDQogCSAqIFR3byBjYXNl czogZWl0aGVyIHdlIHN0ZXBwZWQgb2ZmIHRoZSBlbmQgb2Ygb3VyIG1hc2ss DQpAQCAtMTAxNSw4ICsxMDY2LDYgQEANCiAJcm4gPSBsYXN0Ow0KIAlsYXN0 YiA9IHJuLT5ybl9iaXQ7DQogDQotCS8qIHByaW50Zigicm4gJXAsIGxhc3Ri ICVkXG4iLCBybiwgbGFzdGIpOyovDQotDQogCS8qDQogCSAqIFRoaXMgZ2V0 cyBjb21wbGljYXRlZCBiZWNhdXNlIHdlIG1heSBkZWxldGUgdGhlIG5vZGUN CiAJICogd2hpbGUgYXBwbHlpbmcgdGhlIGZ1bmN0aW9uIGYgdG8gaXQsIHNv IHdlIG5lZWQgdG8gY2FsY3VsYXRlDQpAQCAtMTAyNiw3ICsxMDc1LDYgQEAN CiAJCXJuID0gcm4tPnJuX2xlZnQ7DQogDQogCXdoaWxlICghc3RvcHBpbmcp IHsNCi0JCS8qIHByaW50Zigibm9kZSAlcCAoJWQpXG4iLCBybiwgcm4tPnJu X2JpdCk7ICovDQogCQliYXNlID0gcm47DQogCQkvKiBJZiBhdCByaWdodCBj aGlsZCBnbyBiYWNrIHVwLCBvdGhlcndpc2UsIGdvIHJpZ2h0ICovDQogCQl3 aGlsZSAocm4tPnJuX3BhcmVudC0+cm5fcmlnaHQgPT0gcm4NCkBAIC0xMDM2 LDcgKzEwODQsNiBAQA0KIAkJCS8qIGlmIHdlbnQgdXAgYmV5b25kIGxhc3Qs IHN0b3AgKi8NCiAJCQlpZiAocm4tPnJuX2JpdCA8PSBsYXN0Yikgew0KIAkJ CQlzdG9wcGluZyA9IDE7DQotCQkJCS8qIHByaW50ZigidXAgdG9vIGZhclxu Iik7ICovDQogCQkJCS8qDQogCQkJCSAqIFhYWCB3ZSBzaG91bGQganVtcCB0 byB0aGUgJ1Byb2Nlc3MgbGVhdmVzJw0KIAkJCQkgKiBwYXJ0LCBiZWNhdXNl IHRoZSB2YWx1ZXMgb2YgJ3JuJyBhbmQgJ25leHQnDQpAQCAtMTA2Miw3ICsx MTA5LDYgQEANCiAJCS8qIFByb2Nlc3MgbGVhdmVzICovDQogCQl3aGlsZSAo KHJuID0gYmFzZSkgIT0gMCkgew0KIAkJCWJhc2UgPSBybi0+cm5fZHVwZWRr ZXk7DQotCQkJLyogcHJpbnRmKCJsZWFmICVwXG4iLCBybik7ICovDQogCQkJ aWYgKCEocm4tPnJuX2ZsYWdzICYgUk5GX1JPT1QpDQogCQkJICAgICYmIChl cnJvciA9ICgqZikocm4sIHcpKSkNCiAJCQkJcmV0dXJuIChlcnJvcik7DQpA QCAtMTA3MCw3ICsxMTE2LDYgQEANCiAJCXJuID0gbmV4dDsNCiANCiAJCWlm IChybi0+cm5fZmxhZ3MgJiBSTkZfUk9PVCkgew0KLQkJCS8qIHByaW50Zigi cm9vdCwgc3RvcHBpbmciKTsgKi8NCiAJCQlzdG9wcGluZyA9IDE7DQogCQl9 DQogDQpkaWZmIC11IC1yIC91c3JfZGlmZi9zcmMvc3lzL25ldC9yYWRpeC5o IC9yb3V0ZXIvdXNyL3NyYy9zeXMvbmV0L3JhZGl4LmgNCi0tLSAvdXNyX2Rp ZmYvc3JjL3N5cy9uZXQvcmFkaXguaAkyMDEwLTAzLTIzIDA5OjU4OjU5LjAw MDAwMDAwMCArMDAwMA0KKysrIC9yb3V0ZXIvdXNyL3NyYy9zeXMvbmV0L3Jh ZGl4LmgJMjAxMC0wOS0xMCAyMDoxNTo0Ni4wMDAwMDAwMDAgKzAwMDANCkBA IC0xMTQsNyArMTE0LDcgQEANCiAJCSh2b2lkICp2LCB2b2lkICptYXNrLA0K IAkJICAgICBzdHJ1Y3QgcmFkaXhfbm9kZV9oZWFkICpoZWFkLCBzdHJ1Y3Qg cmFkaXhfbm9kZSBub2Rlc1tdKTsNCiAJc3RydWN0CXJhZGl4X25vZGUgKigq cm5oX2RlbGFkZHIpCS8qIHJlbW92ZSBiYXNlZCBvbiBzb2NrYWRkciAqLw0K LQkJKHZvaWQgKnYsIHZvaWQgKm1hc2ssIHN0cnVjdCByYWRpeF9ub2RlX2hl YWQgKmhlYWQpOw0KKwkJKHZvaWQgKnYsIHZvaWQgKm1hc2ssIHN0cnVjdCBy YWRpeF9ub2RlX2hlYWQgKmhlYWQsIHN0cnVjdCByYWRpeF9ub2RlICpybik7 DQogCXN0cnVjdAlyYWRpeF9ub2RlICooKnJuaF9kZWxwa3QpCS8qIHJlbW92 ZSBiYXNlZCBvbiBwYWNrZXQgaGRyICovDQogCQkodm9pZCAqdiwgdm9pZCAq bWFzaywgc3RydWN0IHJhZGl4X25vZGVfaGVhZCAqaGVhZCk7DQogCXN0cnVj dAlyYWRpeF9ub2RlICooKnJuaF9tYXRjaGFkZHIpCS8qIGxvY2F0ZSBiYXNl ZCBvbiBzb2NrYWRkciAqLw0KQEAgLTE2OCw3ICsxNjgsNyBAQA0KIAkgKnJu X2FkZG1hc2sodm9pZCAqLCBpbnQsIGludCksDQogCSAqcm5fYWRkcm91dGUg KHZvaWQgKiwgdm9pZCAqLCBzdHJ1Y3QgcmFkaXhfbm9kZV9oZWFkICosDQog CQkJc3RydWN0IHJhZGl4X25vZGUgWzJdKSwNCi0JICpybl9kZWxldGUodm9p ZCAqLCB2b2lkICosIHN0cnVjdCByYWRpeF9ub2RlX2hlYWQgKiksDQorCSAq cm5fZGVsZXRlKHZvaWQgKiwgdm9pZCAqLCBzdHJ1Y3QgcmFkaXhfbm9kZV9o ZWFkICosIHN0cnVjdCByYWRpeF9ub2RlICopLA0KIAkgKnJuX2xvb2t1cCAo dm9pZCAqdl9hcmcsIHZvaWQgKm1fYXJnLA0KIAkJICAgICAgICBzdHJ1Y3Qg cmFkaXhfbm9kZV9oZWFkICpoZWFkKSwNCiAJICpybl9tYXRjaCh2b2lkICos IHN0cnVjdCByYWRpeF9ub2RlX2hlYWQgKik7DQpkaWZmIC11IC1yIC91c3Jf ZGlmZi9zcmMvc3lzL25ldC9yYWRpeF9tcGF0aC5jIC9yb3V0ZXIvdXNyL3Ny Yy9zeXMvbmV0L3JhZGl4X21wYXRoLmMNCi0tLSAvdXNyX2RpZmYvc3JjL3N5 cy9uZXQvcmFkaXhfbXBhdGguYwkyMDEwLTA0LTAyIDA1OjAyOjUwLjAwMDAw MDAwMCArMDAwMA0KKysrIC9yb3V0ZXIvdXNyL3NyYy9zeXMvbmV0L3JhZGl4 X21wYXRoLmMJMjAxMC0wOS0xNiAxOToyNDo0NS4wMDAwMDAwMDAgKzAwMDAN CkBAIC0xMTIsNDYgKzExMiwxNCBAQA0KIAkJICogd2UgbmVlZCB0byBjb21w YXJlIHRoZSBpbnRlcmZhY2UgYWRkcmVzcyBiZWNhdXNlDQogCQkgKiBydF9n YXRld2F5IGlzIGEgc3BlY2lhbCBzb2NrYWRkX2RsIHN0cnVjdHVyZQ0KIAkJ ICovDQotCQlpZiAocnQtPnJ0X2dhdGV3YXktPnNhX2ZhbWlseSA9PSBBRl9M SU5LKSB7DQotCQkJaWYgKCFtZW1jbXAocnQtPnJ0X2lmYS0+aWZhX2FkZHIs IGdhdGUsIGdhdGUtPnNhX2xlbikpDQorCQlpZiAocnQtPnJ0X2dhdGV3YXkt PnNhX2xlbiA9PSBnYXRlLT5zYV9sZW4gJiYNCisJCQkhbWVtY21wKHJ0LT5y dF9nYXRld2F5LCBnYXRlLCBnYXRlLT5zYV9sZW4pKQ0KIAkJCQlicmVhazsN Ci0JCX0gZWxzZSB7DQotCQkJaWYgKHJ0LT5ydF9nYXRld2F5LT5zYV9sZW4g PT0gZ2F0ZS0+c2FfbGVuICYmDQotCQkJICAgICFtZW1jbXAocnQtPnJ0X2dh dGV3YXksIGdhdGUsIGdhdGUtPnNhX2xlbikpDQotCQkJCWJyZWFrOw0KLQkJ fQ0KIAl9IHdoaWxlICgocm4gPSBybl9tcGF0aF9uZXh0KHJuKSkgIT0gTlVM TCk7DQogDQogCXJldHVybiAoc3RydWN0IHJ0ZW50cnkgKilybjsNCiB9DQog DQotLyogDQotICogZ28gdGhyb3VnaCB0aGUgY2hhaW4gYW5kIHVubGluayAi cnQiIGZyb20gdGhlIGxpc3QNCi0gKiB0aGUgY2FsbGVyIHdpbGwgZnJlZSAi cnQiDQotICovDQotaW50DQotcnRfbXBhdGhfZGVsZHVwKHN0cnVjdCBydGVu dHJ5ICpoZWFkcnQsIHN0cnVjdCBydGVudHJ5ICpydCkNCi17DQotICAgICAg ICBzdHJ1Y3QgcmFkaXhfbm9kZSAqdCwgKnR0Ow0KLQ0KLSAgICAgICAgaWYg KCFoZWFkcnQgfHwgIXJ0KQ0KLSAgICAgICAgICAgIHJldHVybiAoMCk7DQot ICAgICAgICB0ID0gKHN0cnVjdCByYWRpeF9ub2RlICopaGVhZHJ0Ow0KLSAg ICAgICAgdHQgPSBybl9tcGF0aF9uZXh0KHQpOw0KLSAgICAgICAgd2hpbGUg KHR0KSB7DQotICAgICAgICAgICAgaWYgKHR0ID09IChzdHJ1Y3QgcmFkaXhf bm9kZSAqKXJ0KSB7DQotICAgICAgICAgICAgICAgIHQtPnJuX2R1cGVka2V5 ID0gdHQtPnJuX2R1cGVka2V5Ow0KLSAgICAgICAgICAgICAgICB0dC0+cm5f ZHVwZWRrZXkgPSBOVUxMOw0KLSAgICAJICAgICAgICB0dC0+cm5fZmxhZ3Mg Jj0gflJORl9BQ1RJVkU7DQotCSAgICAgICAgdHRbMV0ucm5fZmxhZ3MgJj0g flJORl9BQ1RJVkU7DQotICAgICAgICAgICAgICAgIHJldHVybiAoMSk7DQot ICAgICAgICAgICAgfQ0KLSAgICAgICAgICAgIHQgPSB0dDsNCi0gICAgICAg ICAgICB0dCA9IHJuX21wYXRoX25leHQoKHN0cnVjdCByYWRpeF9ub2RlICop dCk7DQotICAgICAgICB9DQotICAgICAgICByZXR1cm4gKDApOw0KLX0NCi0N CiAvKg0KICAqIGNoZWNrIGlmIHdlIGhhdmUgdGhlIHNhbWUga2V5L21hc2sv Z2F0ZXdheSBvbiB0aGUgdGFibGUgYWxyZWFkeS4NCiAgKi8NCkBAIC0yNjIs OSArMjMwLDEwIEBADQogcnRhbGxvY19tcGF0aF9maWIoc3RydWN0IHJvdXRl ICpybywgdWludDMyX3QgaGFzaCwgdV9pbnQgZmlibnVtKQ0KIHsNCiAJc3Ry dWN0IHJhZGl4X25vZGUgKnJuMCwgKnJuOw0KLQl1X2ludDMyX3QgbjsNCisJ dV9pbnQzMl90IG4gPSAwOw0KIAlzdHJ1Y3QgcnRlbnRyeSAqcnQ7DQogCWlu dDY0X3Qgd2VpZ2h0Ow0KKwlpbnQ2NF90IGxvd2VzdF93ZWlnaHQ7DQogDQog CS8qDQogCSAqIFhYWCB3ZSBkb24ndCBhdHRlbXB0IHRvIGxvb2t1cCBjYWNo ZWQgcm91dGUgYWdhaW47IHdoYXQgc2hvdWxkDQpAQCAtMjcyLDcgKzI0MSw3 IEBADQogCSAqLw0KIAlpZiAocm8tPnJvX3J0ICYmIHJvLT5yb19ydC0+cnRf aWZwICYmIChyby0+cm9fcnQtPnJ0X2ZsYWdzICYgUlRGX1VQKQ0KIAkgICAg JiYgUlRfTElOS19JU19VUChyby0+cm9fcnQtPnJ0X2lmcCkpDQotCQlyZXR1 cm47CQkJCSANCisJCXJldHVybjsNCiAJcm8tPnJvX3J0ID0gcnRhbGxvYzFf ZmliKCZyby0+cm9fZHN0LCAxLCAwLCBmaWJudW0pOw0KIA0KIAkvKiBpZiB0 aGUgcm91dGUgZG9lcyBub3QgZXhpc3Qgb3IgaXQgaXMgbm90IG11bHRpcGF0 aCwgZG9uJ3QgY2FyZSAqLw0KQEAgLTI4NSwyMCArMjU0LDM0IEBADQogDQog CS8qIGJleW9uZCBoZXJlLCB3ZSB1c2Ugcm4gYXMgdGhlIG1hc3RlciBjb3B5 ICovDQogCXJuMCA9IHJuID0gKHN0cnVjdCByYWRpeF9ub2RlICopcm8tPnJv X3J0Ow0KLQluID0gcm5fbXBhdGhfY291bnQocm4wKTsNCi0NCisJDQorCS8q IGZpbmQgbG93ZXN0IHdlaWdodCByb3V0ZSAqLw0KKwlmb3IgKCBydCA9IChz dHJ1Y3QgcnRlbnRyeSAqKXJuLCB3ZWlnaHQgPSBydC0+cnRfcm14LnJteF93 ZWlnaHQ7IHJuICE9IE5VTEw7IHJuID0gcm5fbXBhdGhfbmV4dCggcm4pKSB7 DQorCQkvKiBYWFggY2hlY2sgaWYgcm91dGUgaXMgdXA/ICovDQorCQlydCA9 IChzdHJ1Y3QgcnRlbnRyeSAqKXJuOw0KKwkJaWYocnQtPnJ0X2ZsYWdzICYg UlRGX1VQKSB7IA0KKwkJCWlmICh3ZWlnaHQgPiBydC0+cnRfcm14LnJteF93 ZWlnaHQpIHsNCisJCQkJd2VpZ2h0ID0gcnQtPnJ0X3JteC5ybXhfd2VpZ2h0 Ow0KKwkJCQluID0gMTsNCisJCQl9IGVsc2UgaWYgKHdlaWdodCA9PSBydC0+ cnRfcm14LnJteF93ZWlnaHQpDQorCQkJCW4rKzsNCisJCX0NCisJfQ0KKwls b3dlc3Rfd2VpZ2h0ID0gd2VpZ2h0Ow0KKwkNCisJLyogc2VsZWN0IG5vdyBv bmUgb2YgdGhlIGxvd2VzdCB3ZWlnaHQgcm91dGVzICovDQogCS8qIGd3IHNl bGVjdGlvbiBieSBNb2R1bG8tTiBIYXNoIChSRkMyOTkxKSBYWFggbmVlZCBp bXByb3ZlbWVudD8gKi8NCiAJaGFzaCArPSBoYXNoaml0dGVyOw0KIAloYXNo ICU9IG47DQotCWZvciAod2VpZ2h0ID0gYWJzKChpbnQzMl90KWhhc2gpLCBy dCA9IHJvLT5yb19ydDsNCi0JICAgICB3ZWlnaHQgPj0gcnQtPnJ0X3JteC5y bXhfd2VpZ2h0ICYmIHJuOyANCi0JICAgICB3ZWlnaHQgLT0gcnQtPnJ0X3Jt eC5ybXhfd2VpZ2h0KSB7DQotCQkNCi0JCS8qIHN0YXkgd2l0aGluIHRoZSBt dWx0aXBhdGggcm91dGVzICovDQotCQlpZiAocm4tPnJuX2R1cGVka2V5ICYm IHJuLT5ybl9tYXNrICE9IHJuLT5ybl9kdXBlZGtleS0+cm5fbWFzaykNCi0J CQlicmVhazsNCi0JCXJuID0gcm4tPnJuX2R1cGVka2V5Ow0KKwlmb3IgKCBy biA9IHJuMCwgbiA9IDA7IHJuICE9IE5VTEw7IHJuID0gcm5fbXBhdGhfbmV4 dCggcm4pKSB7DQogCQlydCA9IChzdHJ1Y3QgcnRlbnRyeSAqKXJuOw0KKwkJ aWYocnQtPnJ0X2ZsYWdzICYgUlRGX1VQKSB7IA0KKwkJCWlmICggcnQtPnJ0 X3JteC5ybXhfd2VpZ2h0ID09IGxvd2VzdF93ZWlnaHQpIHsNCisJCQkJaWYg KG4gPT0gaGFzaCkNCisJCQkJCWJyZWFrOw0KKwkJCQluKys7DQorCQkJfQ0K KwkJfQ0KIAl9DQogCS8qIFhYWCB0cnkgZmlsbGluZyBydF9nd3JvdXRlIGFu ZCBhdm9pZCB1bnJlYWNoYWJsZSBndyAgKi8NCiANCmRpZmYgLXUgLXIgL3Vz cl9kaWZmL3NyYy9zeXMvbmV0L3JvdXRlLmMgL3JvdXRlci91c3Ivc3JjL3N5 cy9uZXQvcm91dGUuYw0KLS0tIC91c3JfZGlmZi9zcmMvc3lzL25ldC9yb3V0 ZS5jCTIwMTAtMDYtMTggMDM6MzE6MzMuMDAwMDAwMDAwICswMDAwDQorKysg L3JvdXRlci91c3Ivc3JjL3N5cy9uZXQvcm91dGUuYwkyMDEwLTA5LTE2IDE5 OjEyOjE4LjAwMDAwMDAwMCArMDAwMA0KQEAgLTg3NSw3ICs4NzUsNyBAQA0K IAkgKiBSZW1vdmUgdGhlIGl0ZW0gZnJvbSB0aGUgdHJlZTsgaXQgc2hvdWxk IGJlIHRoZXJlLA0KIAkgKiBidXQgd2hlbiBjYWxsZXJzIGludm9rZSB1cyBi bGluZGx5IGl0IG1heSBub3QgKHNpZ2gpLg0KIAkgKi8NCi0Jcm4gPSBybmgt PnJuaF9kZWxhZGRyKHJ0X2tleShydCksIHJ0X21hc2socnQpLCBybmgpOw0K KwlybiA9IHJuaC0+cm5oX2RlbGFkZHIocnRfa2V5KHJ0KSwgcnRfbWFzayhy dCksIHJuaCwgTlVMTCk7DQogCWlmIChybiA9PSBOVUxMKSB7DQogCQllcnJv ciA9IEVTUkNIOw0KIAkJZ290byBiYWQ7DQpAQCAtOTEzLDExMiArOTEzLDYg QEANCiAJcmV0dXJuIChlcnJvcik7DQogfQ0KIA0KLSNpZmRlZiBSQURJWF9N UEFUSA0KLXN0YXRpYyBpbnQNCi1ybl9tcGF0aF91cGRhdGUoaW50IHJlcSwg c3RydWN0IHJ0X2FkZHJpbmZvICppbmZvLA0KLSAgICBzdHJ1Y3QgcmFkaXhf bm9kZV9oZWFkICpybmgsIHN0cnVjdCBydGVudHJ5ICoqcmV0X25ydCkNCi17 DQotCS8qDQotCSAqIGlmIHdlIGdvdCBtdWx0aXBhdGggcm91dGVzLCB3ZSBy ZXF1aXJlIHVzZXJzIHRvIHNwZWNpZnkNCi0JICogYSBtYXRjaGluZyBSVEFY X0dBVEVXQVkuDQotCSAqLw0KLQlzdHJ1Y3QgcnRlbnRyeSAqcnQsICpydG8g PSBOVUxMOw0KLQlyZWdpc3RlciBzdHJ1Y3QgcmFkaXhfbm9kZSAqcm47DQot CWludCBlcnJvciA9IDA7DQotDQotCXJuID0gcm5oLT5ybmhfbWF0Y2hhZGRy KGRzdCwgcm5oKTsNCi0JaWYgKHJuID09IE5VTEwpDQotCQlyZXR1cm4gKEVT UkNIKTsNCi0JcnRvID0gcnQgPSBSTlRPUlQocm4pOw0KLQlydCA9IHJ0X21w YXRoX21hdGNoZ2F0ZShydCwgZ2F0ZXdheSk7DQotCWlmIChydCA9PSBOVUxM KQ0KLQkJcmV0dXJuIChFU1JDSCk7DQotCS8qDQotCSAqIHRoaXMgaXMgdGhl IGZpcnN0IGVudHJ5IGluIHRoZSBjaGFpbg0KLQkgKi8NCi0JaWYgKHJ0byA9 PSBydCkgew0KLQkJcm4gPSBybl9tcGF0aF9uZXh0KChzdHJ1Y3QgcmFkaXhf bm9kZSAqKXJ0KTsNCi0JCS8qDQotCQkgKiB0aGVyZSBpcyBhbm90aGVyIGVu dHJ5LCBub3cgaXQncyBhY3RpdmUNCi0JCSAqLw0KLQkJaWYgKHJuKSB7DQot CQkJcnRvID0gUk5UT1JUKHJuKTsNCi0JCQlSVF9MT0NLKHJ0byk7DQotCQkJ cnRvLT5ydF9mbGFncyB8PSBSVEZfVVA7DQotCQkJUlRfVU5MT0NLKHJ0byk7 DQotCQl9IGVsc2UgaWYgKHJ0LT5ydF9mbGFncyAmIFJURl9HQVRFV0FZKSB7 DQotCQkJLyoNCi0JCQkgKiBGb3IgZ2F0ZXdheSByb3V0ZXMsIHdlIG5lZWQg dG8gDQotCQkJICogbWFrZSBzdXJlIHRoYXQgd2Ugd2UgYXJlIGRlbGV0aW5n DQotCQkJICogdGhlIGNvcnJlY3QgZ2F0ZXdheS4gDQotCQkJICogcnRfbXBh dGhfbWF0Y2hnYXRlKCkgZG9lcyBub3QgDQotCQkJICogY2hlY2sgdGhlIGNh c2Ugd2hlbiB0aGVyZSBpcyBvbmx5DQotCQkJICogb25lIHJvdXRlIGluIHRo ZSBjaGFpbi4gIA0KLQkJCSAqLw0KLQkJCWlmIChnYXRld2F5ICYmDQotCQkJ ICAgIChydC0+cnRfZ2F0ZXdheS0+c2FfbGVuICE9IGdhdGV3YXktPnNhX2xl biB8fA0KLQkJCQltZW1jbXAocnQtPnJ0X2dhdGV3YXksIGdhdGV3YXksIGdh dGV3YXktPnNhX2xlbikpKQ0KLQkJCQllcnJvciA9IEVTUkNIOw0KLQkJCWVs c2Ugew0KLQkJCQkvKg0KLQkJCQkgKiByZW1vdmUgZnJvbSB0cmVlIGJlZm9y ZSByZXR1cm5pbmcgaXQNCi0JCQkJICogdG8gdGhlIGNhbGxlcg0KLQkJCQkg Ki8NCi0JCQkJcm4gPSBybmgtPnJuaF9kZWxhZGRyKGRzdCwgbmV0bWFzaywg cm5oKTsNCi0JCQkJS0FTU0VSVChydCA9PSBSTlRPUlQocm4pLCAoInJhZGl4 IG5vZGUgZGlzYXBwZWFyZWQiKSk7DQotCQkJCWdvdG8gZ3dkZWxldGU7DQot CQkJfQ0KLQkJCQ0KLQkJfQ0KLQkJLyoNCi0JCSAqIHVzZSB0aGUgbm9ybWFs IGRlbGV0ZSBjb2RlIHRvIHJlbW92ZQ0KLQkJICogdGhlIGZpcnN0IGVudHJ5 DQotCQkgKi8NCi0JCWlmIChyZXEgIT0gUlRNX0RFTEVURSkgDQotCQkJZ290 byBub25kZWxldGU7DQotDQotCQllcnJvciA9IEVOT0VOVDsNCi0JCWdvdG8g ZG9uZTsNCi0JfQ0KLQkJDQotCS8qDQotCSAqIGlmIHRoZSBlbnRyeSBpcyAy bmQgYW5kIG9uIHVwDQotCSAqLw0KLQlpZiAoKHJlcSA9PSBSVE1fREVMRVRF KSAmJiAhcnRfbXBhdGhfZGVsZHVwKHJ0bywgcnQpKQ0KLQkJcGFuaWMgKCJy dHJlcXVlc3QxOiBydF9tcGF0aF9kZWxkdXAiKTsNCi1nd2RlbGV0ZToNCi0J UlRfTE9DSyhydCk7DQotCVJUX0FERFJFRihydCk7DQotCWlmIChyZXEgPT0g UlRNX0RFTEVURSkgew0KLQkJcnQtPnJ0X2ZsYWdzICY9IH5SVEZfVVA7DQot CQkvKg0KLQkJICogT25lIG1vcmUgcnRlbnRyeSBmbG9hdGluZyBhcm91bmQg dGhhdCBpcyBub3QNCi0JCSAqIGxpbmtlZCB0byB0aGUgcm91dGluZyB0YWJs ZS4gcnR0cmFzaCB3aWxsIGJlIGRlY3JlbWVudGVkDQotCQkgKiB3aGVuIFJU RlJFRShydCkgaXMgZXZlbnR1YWxseSBjYWxsZWQuDQotCQkgKi8NCi0JCVZf cnR0cmFzaCsrOw0KLQl9DQotCQ0KLW5vbmRlbGV0ZToNCi0JaWYgKHJlcSAh PSBSVE1fREVMRVRFKQ0KLQkJcGFuaWMoInVucmVjb2duaXplZCByZXF1ZXN0 ICVkIiwgcmVxKTsNCi0JDQotDQotCS8qDQotCSAqIElmIHRoZSBjYWxsZXIg d2FudHMgaXQsIHRoZW4gaXQgY2FuIGhhdmUgaXQsDQotCSAqIGJ1dCBpdCdz IHVwIHRvIGl0IHRvIGZyZWUgdGhlIHJ0ZW50cnkgYXMgd2Ugd29uJ3QgYmUN Ci0JICogZG9pbmcgaXQuDQotCSAqLw0KLQlpZiAocmV0X25ydCkgew0KLQkJ KnJldF9ucnQgPSBydDsNCi0JCVJUX1VOTE9DSyhydCk7DQotCX0gZWxzZQ0K LQkJUlRGUkVFX0xPQ0tFRChydCk7DQotZG9uZToNCi0JcmV0dXJuIChlcnJv cik7DQotfQ0KLSNlbmRpZg0KLQ0KIGludA0KIHJ0cmVxdWVzdDFfZmliKGlu dCByZXEsIHN0cnVjdCBydF9hZGRyaW5mbyAqaW5mbywgc3RydWN0IHJ0ZW50 cnkgKipyZXRfbnJ0LA0KIAkJCQl1X2ludCBmaWJudW0pDQpAQCAtMTA1OCwy OCArOTUyLDMwIEBADQogDQogCXN3aXRjaCAocmVxKSB7DQogCWNhc2UgUlRN X0RFTEVURToNCisJICAgICAgICBpZiAoKHJuID0gcm5oLT5ybmhfbG9va3Vw KGRzdCwgbmV0bWFzaywgcm5oKSkgPT0gTlVMTCkNCisJICAgICAgICAgICAg ICAgIHNlbmRlcnIoRVNSQ0gpOw0KKwkJcnQgPSBSTlRPUlQocm4pOw0KICNp ZmRlZiBSQURJWF9NUEFUSA0KLQkJaWYgKHJuX21wYXRoX2NhcGFibGUocm5o KSkgew0KLQkJCWVycm9yID0gcm5fbXBhdGhfdXBkYXRlKHJlcSwgaW5mbywg cm5oLCByZXRfbnJ0KTsNCi0JCQkvKg0KLQkJCSAqICJiYWQiIGhvbGRzIHRy dWUgZm9yIHRoZSBzdWNjZXNzIGNhc2UNCi0JCQkgKiBhcyB3ZWxsDQotCQkJ ICovDQotCQkJaWYgKGVycm9yICE9IEVOT0VOVCkNCi0JCQkJZ290byBiYWQ7 DQotCQkJZXJyb3IgPSAwOw0KLQkJfQ0KKyAgICAgICAgICAgICAgICAvKg0K KyAgICAgICAgICAgICAgICAgKiBpZiB3ZSBnb3QgbXVsdGlwYXRoIHJvdXRl cywgd2UgcmVxdWlyZSB1c2VycyB0byBzcGVjaWZ5DQorICAgICAgICAgICAg ICAgICAqIGEgbWF0Y2hpbmcgUlRBWF9HQVRFV0FZLg0KKyAgICAgICAgICAg ICAgICAgKi8NCisgICAgICAgICAgICAgICAgaWYgKHJuX21wYXRoX2NhcGFi bGUocm5oKSkgew0KKyAgICAgICAgICAgICAgICAgICAgICAgIHJ0ID0gcnRf bXBhdGhfbWF0Y2hnYXRlKCBydCwgZ2F0ZXdheSk7DQorICAgICAgICAgICAg ICAgICAgICAgICAgcm4gPSAoc3RydWN0IHJhZGl4X25vZGUgKilydDsNCisg ICAgICAgICAgICAgICAgICAgICAgICBpZiAoIXJ0KQ0KKyAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgc2VuZGVycihFU1JDSCk7DQorICAgICAg ICAgICAgICAgIH0NCiAjZW5kaWYNCiAJCS8qDQogCQkgKiBSZW1vdmUgdGhl IGl0ZW0gZnJvbSB0aGUgdHJlZSBhbmQgcmV0dXJuIGl0Lg0KIAkJICogQ29t cGxhaW4gaWYgaXQgaXMgbm90IHRoZXJlIGFuZCBkbyBubyBtb3JlIHByb2Nl c3NpbmcuDQogCQkgKi8NCi0JCXJuID0gcm5oLT5ybmhfZGVsYWRkcihkc3Qs IG5ldG1hc2ssIHJuaCk7DQorCQlybiA9IHJuaC0+cm5oX2RlbGFkZHIoZHN0 LCBuZXRtYXNrLCBybmgsIHJuKTsNCiAJCWlmIChybiA9PSBOVUxMKQ0KIAkJ CXNlbmRlcnIoRVNSQ0gpOw0KIAkJaWYgKHJuLT5ybl9mbGFncyAmIChSTkZf QUNUSVZFIHwgUk5GX1JPT1QpKQ0KIAkJCXBhbmljICgicnRyZXF1ZXN0IGRl bGV0ZSIpOw0KLQkJcnQgPSBSTlRPUlQocm4pOw0KIAkJUlRfTE9DSyhydCk7 DQogCQlSVF9BRERSRUYocnQpOw0KIAkJcnQtPnJ0X2ZsYWdzICY9IH5SVEZf VVA7DQpAQCAtMTQ3NCwxMCArMTM3MCw5IEBADQogCQkJICAgIFJOVE9SVChy biktPnJ0X2lmYSAhPSBpZmEgfHwNCiAJCQkgICAgIXNhX2VxdWFsKChzdHJ1 Y3Qgc29ja2FkZHIgKilybi0+cm5fa2V5LCBkc3QpKTsNCiAJCQlSQURJWF9O T0RFX0hFQURfVU5MT0NLKHJuaCk7DQotCQkJaWYgKGVycm9yKSB7DQorCQkJ aWYgKGVycm9yKQ0KIAkJCQkvKiB0aGlzIGlzIG9ubHkgYW4gZXJyb3IgaWYg YmFkIG9uIEFMTCB0YWJsZXMgKi8NCiAJCQkJY29udGludWU7DQotCQkJfQ0K IAkJfQ0KIAkJLyoNCiAJCSAqIERvIHRoZSBhY3R1YWwgcmVxdWVzdA0KZGlm ZiAtdSAtciAvdXNyX2RpZmYvc3JjL3N5cy9uZXRpbmV0L2luLmMgL3JvdXRl ci91c3Ivc3JjL3N5cy9uZXRpbmV0L2luLmMNCi0tLSAvdXNyX2RpZmYvc3Jj L3N5cy9uZXRpbmV0L2luLmMJMjAxMC0wOS0wNyAxMzoxMDo0Ni4wMDAwMDAw MDAgKzAwMDANCisrKyAvcm91dGVyL3Vzci9zcmMvc3lzL25ldGluZXQvaW4u YwkyMDEwLTA5LTE2IDE5OjE4OjQ2LjAwMDAwMDAwMCArMDAwMA0KQEAgLTEz NzQsMTIgKzEzNzQsNDUgQEANCiBpbl9sbHRhYmxlX3J0Y2hlY2soc3RydWN0 IGlmbmV0ICppZnAsIHVfaW50IGZsYWdzLCBjb25zdCBzdHJ1Y3Qgc29ja2Fk ZHIgKmwzYWRkcikNCiB7DQogCXN0cnVjdCBydGVudHJ5ICpydDsNCisjaWZk ZWYgUkFESVhfTVBBVEgNCisJaW50NjRfdCB3ZWlnaHQ7DQorCXN0cnVjdCBy dGVudHJ5ICpydDA7DQorCWludDMyX3QgZm91bmQgPSAwOw0KKyNlbmRpZg0K IA0KIAlLQVNTRVJUKGwzYWRkci0+c2FfZmFtaWx5ID09IEFGX0lORVQsDQog CSAgICAoInNpbl9mYW1pbHkgJWQiLCBsM2FkZHItPnNhX2ZhbWlseSkpOw0K IA0KIAkvKiBYWFggcnRhbGxvYzEgc2hvdWxkIHRha2UgYSBjb25zdCBwYXJh bSAqLw0KIAlydCA9IHJ0YWxsb2MxKF9fREVDT05TVChzdHJ1Y3Qgc29ja2Fk ZHIgKiwgbDNhZGRyKSwgMCwgMCk7DQorDQorI2lmZGVmIFJBRElYX01QQVRI DQorCXJ0MCA9IHJ0Ow0KKyAgICAgICAgaWYgKChydCAhPSBOVUxMKSAmJiAo IHJuX21wYXRoX25leHQoKHN0cnVjdCByYWRpeF9ub2RlICopcnQpICE9IE5V TEwpKSB7DQorCQkvKiBjaGVjayBpZiB0aGVyZSBhcmUgb3RoZXIsIG1hdGNo aW5nIHJvdXRlcyAqLw0KKwkJLyogZmluZCBsb3dlc3Qgd2VpZ2h0IHJvdXRl ICovDQorCQlmb3IgKCB3ZWlnaHQgPSBydC0+cnRfcm14LnJteF93ZWlnaHQ7 IHJ0ICE9IE5VTEw7IHJ0ID0gKHN0cnVjdCBydGVudHJ5ICopcm5fbXBhdGhf bmV4dCggKHN0cnVjdCByYWRpeF9ub2RlICopcnQpKSB7DQorCQkJaWYocnQt PnJ0X2ZsYWdzICYgUlRGX1VQKSB7DQorCQkJCWlmICh3ZWlnaHQgPiBydC0+ cnRfcm14LnJteF93ZWlnaHQpDQorCQkJCQl3ZWlnaHQgPSBydC0+cnRfcm14 LnJteF93ZWlnaHQ7DQorCQkJfQ0KKwkJfQ0KKw0KKwkJLyogZmluZCBub3cg b25lIG5vbiBnYXRld2F5IHJvdXRlIHdpdGggbG93ZXN0IHdlaWdodCAqLw0K KwkJZm9yICggcnQgPSBydDA7IHJ0ICE9IE5VTEw7IHJ0ID0gKHN0cnVjdCBy dGVudHJ5ICopcm5fbXBhdGhfbmV4dCggKHN0cnVjdCByYWRpeF9ub2RlICop cnQpKSB7DQorCQkJaWYocnQtPnJ0X2ZsYWdzICYgUlRGX1VQKSB7DQorCQkJ CWlmICgod2VpZ2h0ID09IHJ0LT5ydF9ybXgucm14X3dlaWdodCkgJiYgIShy dC0+cnRfZmxhZ3MgJiBSVEZfR0FURVdBWSkpIHsNCisJCQkJCWZvdW5kID0g MTsNCisJCQkJCWJyZWFrOw0KKwkJCQl9DQorCQkJfQ0KKwkJfQ0KKwkJaWYg KGZvdW5kID09IDApDQorCQkJcnQgPSBOVUxMOw0KKwl9DQorCQ0KKyNlbmRp ZgkNCisNCiAJaWYgKHJ0ID09IE5VTEwgfHwgKCEoZmxhZ3MgJiBMTEVfUFVC KSAmJg0KIAkJCSAgICgocnQtPnJ0X2ZsYWdzICYgUlRGX0dBVEVXQVkpIHx8 IA0KIAkJCSAgICAocnQtPnJ0X2lmcCAhPSBpZnApKSkpIHsNCkBAIC0xMzg3 LDExICsxNDIwLDIxIEBADQogCQlsb2coTE9HX0lORk8sICJJUHY0IGFkZHJl c3M6IFwiJXNcIiBpcyBub3Qgb24gdGhlIG5ldHdvcmtcbiIsDQogCQkgICAg aW5ldF9udG9hKCgoY29uc3Qgc3RydWN0IHNvY2thZGRyX2luICopbDNhZGRy KS0+c2luX2FkZHIpKTsNCiAjZW5kaWYNCisjaWZkZWYgUkFESVhfTVBBVEgN CisJCWlmIChydDAgIT0gTlVMTCkNCisJCQlSVEZSRUVfTE9DS0VEKHJ0MCk7 DQorI2Vsc2UNCiAJCWlmIChydCAhPSBOVUxMKQ0KIAkJCVJURlJFRV9MT0NL RUQocnQpOw0KKyNlbmRpZg0KIAkJcmV0dXJuIChFSU5WQUwpOw0KIAl9DQor I2lmZGVmIFJBRElYX01QQVRIDQorCVJURlJFRV9MT0NLRUQocnQwKTsNCisj ZWxzZQ0KIAlSVEZSRUVfTE9DS0VEKHJ0KTsNCisjZW5kaWYNCisNCiAJcmV0 dXJuIDA7DQogfQ0KIA0KQEAgLTE0MjQsNyArMTQ2Nyw3IEBADQogCWlmIChs bGUgPT0gTlVMTCkgew0KICNpZmRlZiBESUFHTk9TVElDDQogCQlpZiAoZmxh Z3MgJiBMTEVfREVMRVRFKQ0KLQkJCWxvZyhMT0dfSU5GTywgImludGVyZmFj ZSBhZGRyZXNzIGlzIG1pc3NpbmcgZnJvbSBjYWNoZSA9ICVwICBpbiBkZWxl dGVcbiIsIGxsZSk7CQ0KKwkJCWxvZyhMT0dfSU5GTywgImludGVyZmFjZSBh ZGRyZXNzIGlzIG1pc3NpbmcgZnJvbSBjYWNoZSA9ICVwICBpbiBkZWxldGVc biIsIGxsZSk7DQogI2VuZGlmDQogCQlpZiAoIShmbGFncyAmIExMRV9DUkVB VEUpKQ0KIAkJCXJldHVybiAoTlVMTCk7DQpkaWZmIC11IC1yIC91c3JfZGlm Zi9zcmMvc3lzL25ldGluZXQvaXBmdy9pcF9md190YWJsZS5jIC9yb3V0ZXIv dXNyL3NyYy9zeXMvbmV0aW5ldC9pcGZ3L2lwX2Z3X3RhYmxlLmMNCi0tLSAv dXNyX2RpZmYvc3JjL3N5cy9uZXRpbmV0L2lwZncvaXBfZndfdGFibGUuYwky MDEwLTAzLTIzIDA5OjU4OjU5LjAwMDAwMDAwMCArMDAwMA0KKysrIC9yb3V0 ZXIvdXNyL3NyYy9zeXMvbmV0aW5ldC9pcGZ3L2lwX2Z3X3RhYmxlLmMJMjAx MC0wOC0yNiAxMjo0NzoyNy4wMDAwMDAwMDAgKzAwMDANCkBAIC0xMzcsNyAr MTM3LDcgQEANCiAJbWFzay5zaW5fYWRkci5zX2FkZHIgPSBodG9ubChtbGVu ID8gfigoMSA8PCAoMzIgLSBtbGVuKSkgLSAxKSA6IDApOw0KIAlzYS5zaW5f YWRkci5zX2FkZHIgPSBhZGRyICYgbWFzay5zaW5fYWRkci5zX2FkZHI7DQog CUlQRldfV0xPQ0soY2gpOw0KLQllbnQgPSAoc3RydWN0IHRhYmxlX2VudHJ5 ICopcm5oLT5ybmhfZGVsYWRkcigmc2EsICZtYXNrLCBybmgpOw0KKwllbnQg PSAoc3RydWN0IHRhYmxlX2VudHJ5ICopcm5oLT5ybmhfZGVsYWRkcigmc2Es ICZtYXNrLCBybmgsIE5VTEwpOw0KIAlpZiAoZW50ID09IE5VTEwpIHsNCiAJ CUlQRldfV1VOTE9DSyhjaCk7DQogCQlyZXR1cm4gKEVTUkNIKTsNCkBAIC0x NTQsNyArMTU0LDcgQEANCiAJc3RydWN0IHRhYmxlX2VudHJ5ICplbnQ7DQog DQogCWVudCA9IChzdHJ1Y3QgdGFibGVfZW50cnkgKikNCi0JICAgIHJuaC0+ cm5oX2RlbGFkZHIocm4tPnJuX2tleSwgcm4tPnJuX21hc2ssIHJuaCk7DQor CSAgICBybmgtPnJuaF9kZWxhZGRyKHJuLT5ybl9rZXksIHJuLT5ybl9tYXNr LCBybmgsIE5VTEwpOw0KIAlpZiAoZW50ICE9IE5VTEwpDQogCQlmcmVlKGVu dCwgTV9JUEZXX1RCTCk7DQogCXJldHVybiAoMCk7DQpkaWZmIC11IC1yIC91 c3JfZGlmZi9zcmMvc3lzL25ldGluZXQvcmF3X2lwLmMgL3JvdXRlci91c3Iv c3JjL3N5cy9uZXRpbmV0L3Jhd19pcC5jDQotLS0gL3Vzcl9kaWZmL3NyYy9z eXMvbmV0aW5ldC9yYXdfaXAuYwkyMDEwLTA4LTI3IDE4OjUwOjEyLjAwMDAw MDAwMCArMDAwMA0KKysrIC9yb3V0ZXIvdXNyL3NyYy9zeXMvbmV0aW5ldC9y YXdfaXAuYwkyMDEwLTA5LTExIDAyOjU0OjQ1LjAwMDAwMDAwMCArMDAwMA0K QEAgLTc1NSw2ICs3NTUsOCBAQA0KIAkJaWYgKGVyciA9PSAwKQ0KIAkJCWlh LT5pYV9mbGFncyB8PSBJRkFfUk9VVEU7DQogCQllcnIgPSBpZmFfYWRkX2xv b3BiYWNrX3JvdXRlKChzdHJ1Y3QgaWZhZGRyICopaWEsIHNhKTsNCisJCWlm IChlcnIgPT0gMCkNCisJCSAgICAgICAgaWEtPmlhX2ZsYWdzIHw9IElGQV9S VFNFTEY7DQogCQlpZmFfZnJlZSgmaWEtPmlhX2lmYSk7DQogCQlicmVhazsN CiAJfQ0K --168430090-1874193526-1284665933=:5369-- From owner-freebsd-net@FreeBSD.ORG Thu Sep 16 20:08:06 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD63D1065786; Thu, 16 Sep 2010 20:08:06 +0000 (UTC) (envelope-from ddesimone@verio.net) Received: from relay1-bcrtfl2.verio.net (relay1-bcrtfl2.verio.net [131.103.218.142]) by mx1.freebsd.org (Postfix) with ESMTP id 5B7E58FC19; Thu, 16 Sep 2010 20:08:06 +0000 (UTC) Received: from iad-wprd-xchw02.corp.verio.net (unknown [198.87.7.165]) by relay1-bcrtfl2.verio.net (Postfix) with ESMTP id CD590B03822F; Thu, 16 Sep 2010 15:49:08 -0400 (EDT) Thread-Index: ActV2DoAm1/fg9BSQoyUyB7mUocF7w== Received: from dllstx1-8sst9f1.corp.verio.net ([10.144.2.52]) by iad-wprd-xchw02.corp.verio.net over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675); Thu, 16 Sep 2010 15:49:07 -0400 Received: by dllstx1-8sst9f1.corp.verio.net (sSMTP sendmail emulation); Thu, 16 Sep 2010 14:49:06 -0500 Date: Thu, 16 Sep 2010 14:49:05 -0500 Content-Transfer-Encoding: 7bit From: "David DeSimone" To: "Tom Judge" Content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.4657 Importance: normal Priority: normal Message-ID: <20100916194905.GR4928@verio.net> Mail-Followup-To: Tom Judge , freebsd-net@freebsd.org, bz@freebsd.org References: <273436110.20100916180053@gmail.com> <4C9232F9.70206@tomjudge.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <4C9232F9.70206@tomjudge.com> Precedence: bulk User-Agent: Mutt/1.5.20 (2009-06-14) X-OriginalArrivalTime: 16 Sep 2010 19:49:07.0395 (UTC) FILETIME=[395E0130:01CB55D8] Cc: freebsd-net@freebsd.org, bz@freebsd.org Subject: Re: Strange FreeBSD behavior when trying to forward beetween ipsec crypted gif's. May be a problem with ICMP unreach packets at all X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 20:08:06 -0000 Tom Judge wrote: > > > Question: Why FW2 does not send ICMP need-fragment-but-DF-set > > message to HostB ? > > If you take a look at icmp_error() in sys/netinet/ip_icmp.c you will > see that icmp errors are not sent for packets that have been > previously been decrypted by IPSec. I have a feeling that this is > because icmp_reflect() will not use the correct output path but I may > be and most probably am wrong. Bjoern (CC'd) can probably shed more > light on it. It is not done because the resulting ICMP packets may very well end up being sent to an unrelated host or router which has no SA policy defined, meaning that the ICMP would be sent unencrypted. This would expose some packet data in the payload of the ICMP message that had been previously sent as securely encrypted, giving an attacker some known-plaintext data to use in attempting to guess session keys, among other information exposed. -- David DeSimone == Network Admin == fox@verio.net "I don't like spinach, and I'm glad I don't, because if I liked it I'd eat it, and I just hate it." -- Clarence Darrow This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio, Inc. makes no warranty that this email is error or virus free. Thank you. From owner-freebsd-net@FreeBSD.ORG Fri Sep 17 08:15:16 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 280C51065670 for ; Fri, 17 Sep 2010 08:15:16 +0000 (UTC) (envelope-from freebsd-net@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id D21E78FC0A for ; Fri, 17 Sep 2010 08:15:15 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1OwW6B-0003lA-W4 for freebsd-net@freebsd.org; Fri, 17 Sep 2010 10:15:11 +0200 Received: from nuclight.avtf.net ([217.29.94.29]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 17 Sep 2010 10:15:11 +0200 Received: from vadim_nuclight by nuclight.avtf.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 17 Sep 2010 10:15:11 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Vadim Goncharov Date: Fri, 17 Sep 2010 08:15:00 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 27 Message-ID: References: <4C8E0C1E.2020707@networx.ch> <20100915151632.E31898@maildrop.int.zabbadoz.net> <4C90EAB7.2000902@networx.ch> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: nuclight.avtf.net X-Comment-To: Andre Oppermann User-Agent: slrn/0.9.9p1 (FreeBSD) Cc: freebsd-current@freebsd.org Subject: Re: TCP loopback socket fusing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 08:15:16 -0000 Hi Andre Oppermann! On Wed, 15 Sep 2010 17:48:07 +0200; Andre Oppermann wrote about 'Re: TCP loopback socket fusing': >> 3 If properly doing this for TCP, we should probably also do it for >> other protocols. > UNIX domain sockets already do this. This implementation is particular > for TCP and only touches the protocol specific parts. It's not done at > the socket layer. For UDP it's not that easy to do as most UDP connections > are one-off packets and no permanent binding between two sockets exists. > For SCTP I don't know. From glancing over the code it seems they have, > at least partially, their own socket buffer code. How difficult a fused > socket there would be I can't say. This hack is for TCP only, if an application author chooses a non-TCP protocol for loopback, then he is already aware of Unix domain sockets and have reasons for protocol he have chosen to be not PF_LOCAL. And hacking those protocols adds another piece of complexity to maintenance. In fact, care should be taken to consider non-fusing way of operation as primary because packetization is useful not just for filtering SYNs. BTW, I have heard once man needing something like tcpdump for Unix socket. Is something convenient like this is possible?.. -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org][LJ:/nuclight] From owner-freebsd-net@FreeBSD.ORG Fri Sep 17 08:19:59 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1EE3C106566C for ; Fri, 17 Sep 2010 08:19:59 +0000 (UTC) (envelope-from vl.varlog@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9E7C48FC16 for ; Fri, 17 Sep 2010 08:19:58 +0000 (UTC) Received: by eyx24 with SMTP id 24so1134755eyx.13 for ; Fri, 17 Sep 2010 01:19:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:date:from:x-priority :message-id:to:cc:subject:in-reply-to:references:mime-version :content-type:content-transfer-encoding; bh=PXisFd8CuuDJ/HSKqTJS8rGHB+yWuVTTGNA7c1gkgsE=; b=fnqCaj5+vxfzu16gXVl1H/n6paaiG/DB0M7lEFNRkHwphgIAqgqNPgiCoaez4aipNm YV1Wysqy3AQlZoaDc+7nfJKtnqVAEArezQ9qzuuc5E9XkFpRO3xligyQsvZbzQfrEET5 cPl7EQZ/NkvCWhwy45t50NdWFnVEXrOVgUJjM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:x-priority:message-id:to:cc:subject:in-reply-to :references:mime-version:content-type:content-transfer-encoding; b=KCwVCK6cQL75UCkWYUU39ciRDrr+Q0l1CGcP/nawXaVDUPHFgGfkgUmudWPSM61mjh +43gi5mR9rVR8KKv3pV2VK1EdNSX3DqdrBKbxWYAEmEi5NzmCNq2J8ETvXXJmKH23pgG R5FJRd68MooQG7kOQvt8NvhD3qAppW/iCG3T0= Received: by 10.213.28.131 with SMTP id m3mr6402527ebc.67.1284711597473; Fri, 17 Sep 2010 01:19:57 -0700 (PDT) Received: from v-grigorov-xp.mail.msk ([195.218.191.171]) by mx.google.com with ESMTPS id a48sm5340916eei.1.2010.09.17.01.19.53 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 17 Sep 2010 01:19:54 -0700 (PDT) Date: Fri, 17 Sep 2010 12:18:57 +0400 From: Vladimir Grigorov X-Priority: 3 (Normal) Message-ID: <1307024327.20100917121857@gmail.com> To: Tom Judge In-Reply-To: <4C923353.7090801@tomjudge.com> References: <4C923353.7090801@tomjudge.com> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org Subject: Re: Fwd: Re: Strange FreeBSD behavior when trying to forward beetween ipsec crypted gif's. May be a problem with ICMP unreach packets at all X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 08:19:59 -0000 greets all > If you take a look at icmp_error() in sys/netinet/ip_icmp.c you will see > that icmp errors are not sent for packets that have been previously been > decrypted by IPSec. =20 May be some misunderstandings happens. I have gif and ipsec. IPSEC mode = is transport, that means, traffic encrypted only between gif's=20 outer addresses. As result, traffic in gif encrypted by encrypting ipip= container. But I can view traffic on gif by tcpdump as on=20 regular interfaces. E.g. gif's inner traffic not processed by ipsec at all From owner-freebsd-net@FreeBSD.ORG Fri Sep 17 08:22:32 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3BC16106566C for ; Fri, 17 Sep 2010 08:22:32 +0000 (UTC) (envelope-from alexpalias-bsdnet@yahoo.com) Received: from n7.bullet.re3.yahoo.com (n7.bullet.re3.yahoo.com [68.142.237.92]) by mx1.freebsd.org (Postfix) with SMTP id AF3F48FC0A for ; Fri, 17 Sep 2010 08:22:31 +0000 (UTC) Received: from [68.142.237.90] by n7.bullet.re3.yahoo.com with NNFMP; 17 Sep 2010 08:09:28 -0000 Received: from [66.196.114.79] by t6.bullet.re3.yahoo.com with NNFMP; 17 Sep 2010 08:09:28 -0000 Received: from [127.0.0.1] by omp308.mail.re3.yahoo.com with NNFMP; 17 Sep 2010 08:09:28 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 45632.55557.bm@omp308.mail.re3.yahoo.com Received: (qmail 42951 invoked by uid 60001); 17 Sep 2010 08:09:27 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1284710967; bh=xQXtlim+22DPsZg5ztB0ooo2QZs4DXi9wjNHeCfHYSc=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=o9mTY26a9pBYKYmEDZqtfxm4Eab2WN+lNlz9ps8zJXv3JT+WJRVJmrdipMDzw7U4mQIGDCds2S5MgyLK9NEYzqfMDUeC5fKqUFb6dyJuJRGfrJkK3KZlstBMvTJAoFZ70uLKtr20dQTyB0dGQQYB1eSHmOw2Bs6jW5i8dI040kU= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=v3quPyU5lltDit3B5ZqLKRqHZeuX36DLbzNOC4M5TqFA9Rm7SaJEjy3siBm8i7FjY27fNL894Org3qTn7fqBgerh1wX52J+J2XNNDlr9yJG0HMPuJ40MnrZi3Af5rS9U0sTffp3vJ4AxllVcpRiGNQwLH+HtkEZVh/fJ9MamARs=; Message-ID: <885931.42781.qm@web56406.mail.re3.yahoo.com> X-YMail-OSG: slQZi1EVM1l59gNFsJExgPYZ2p4kjXNwHttGHHTh_Z6cjz3 H1LzTMg.vx_c4FdYay7RP9GIEy0k6r907hoTPUOxXci1KzNkXP_4zuDsXKde QfEC0nSO0QKO4es3x.xhSj7QZsKZI96A2aW9DhaXV.gsu7EDH.r_J_IWL2k8 e7us5ylXJqHEcIvNk_Qx6uaAag3RFwi_F4lahLSFVOS.SaA8HZzj4parPqS8 TVxpohuOBfk2sZxLnENKlA3QirhXlTec4ci72YSfQtmeSSE75MOduQq3L1lq 47rsSsulvu3Ej1X76Pl0Ou3Zv8Du5l0fsiaHl4HhNDwJ93yyxGYIbXtE- Received: from [10.11.100.2] by web56406.mail.re3.yahoo.com via HTTP; Fri, 17 Sep 2010 01:09:27 PDT X-Mailer: YahooMailClassic/11.4.9 YahooMailWebService/0.8.105.279950 Date: Fri, 17 Sep 2010 01:09:27 -0700 (PDT) From: Alexandru Popa To: freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="0-303701128-1284710967=:42781" Subject: em (e1000) card lockup on 8.1-RELEASE, plus rebooting X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: alexpalias-bsdnet@yahoo.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 08:22:32 -0000 --0-303701128-1284710967=:42781 Content-Type: text/plain; charset=us-ascii Hello. We're experiencing strange issues in a server with 6 em ports (2 pci-express dual-port cards and 2 ports on the motherboard). em0 / em1 are on a dual-port gigabit card, and in a few days we've experienced this issue first with em1, then with em0 (across different reboots): The em card stops generating interrupts, and thus it stops receiving packets. After that, it only registers "input errors". All other 5 em ports seem OK. Trying to reinit the card with something like: server# ifconfig em1 down;ifconfig em1 up; ls ...results in the server rebooting. Now, our setup is a bit complicated. em interfaces aren't used for IP directly, but they have a few VLANS each. We have in /boot/loader.conf: autoboot_delay="3" hw.em.rxd=4096 hw.em.txd=4096 net.isr.maxthreads=8 I've loaded pf and netgraph, more precisely ng_ether and ng_netflow. The netgraph config looks something like (with vlan numbers and IPs changed): /usr/sbin/ngctl -f- <<-SEQ mkpeer vlan111: netflow lower iface0 name vlan111:lower netflow connect vlan111: netflow: upper out0 connect vlan222: netflow: lower iface1 connect vlan222: netflow: upper out1 connect vlan333: netflow: lower iface2 connect vlan333: netflow: upper out2 connect vlan444: netflow: lower iface3 connect vlan444: netflow: upper out3 msg netflow: setconfig { iface = 0 conf = 3 } msg netflow: setconfig { iface = 1 conf = 3 } msg netflow: setconfig { iface = 2 conf = 3 } msg netflow: setconfig { iface = 3 conf = 3 } msg netflow: settimeouts { inactive = 15 active = 150 } mkpeer netflow: ksocket export inet/dgram/udp msg netflow:export connect inet/1.1.1.1:1234 SEQ pf uses a few rules on two vlans, and has "set skip on em" "set skip on vlan111" etc (only enabled on two vlans, and we don't use netflow on those). The machine is an external router, so it has a full BGP routing table (330k routes). Other tuning: rx,tx (normal and abs) int delays are all 0, for all cards. /etc/sysctl.conf: net.inet.ip.fastforwarding=1 # I know the next one is useless with flowtable.enable=0 net.inet.flowtable.nmbflows=1000000 net.inet.flowtable.enable=0 net.inet.ip.intr_queue_maxlen=4096 net.route.netisr_maxqlen=1024 net.inet.ip.redirect=0 net.inet.ip.process_options=0 Some info from when the system was running, but with em1 hung: sysctl net.em.1.stats=1: em1: Excessive collisions = 0 em1: Sequence errors = 0 em1: Defer count = 0 em1: Missed Packets = 714538 em1: Receive No Buffers = 20793 em1: Receive Length Errors = 0 em1: Receive errors = 0 em1: Crc errors = 0 em1: Alignment errors = 0 em1: Collision/Carrier extension errors = 0 em1: watchdog timeouts = 0 em1: XON Rcvd = 0 em1: XON Xmtd = 0 em1: XOFF Rcvd = 0 em1: XOFF Xmtd = 0 em1: Good Packets Rcvd = 2806530411 em1: Good Packets Xmtd = 2681378150 em1: TSO Contexts Xmtd = 0 em1: TSO Contexts Failed = 0 sysctl net.em.1.debug=1: em1: Adapter hardware address = 0xffffff80005e4590 em1: CTRL = 0x400c0241 RCTL = 0x4008002 em1: Packet buffer = Tx=16k Rx=32k em1: Flow control watermarks high = 30720 low = 29220 em1: tx_int_delay = 0, tx_abs_int_delay = 0 em1: rx_int_delay = 0, rx_abs_int_delay = 0 em1: Queue(0) tdh = 1805, tdt = 1805 em1: TX(0) no descriptors avail event = 0 em1: TX(0) MSIX IRQ Handled = 0 em1: Num Tx descriptors avail = 4096 em1: Tx Descriptors not avail1 = 0 em1: RX(0) MSIX IRQ Handled = 0 em1: hw rdh = 309, hw rdt = 309 em1: Std mbuf failed = 0 em1: Std mbuf cluster failed = 0 em1: Driver dropped packets = 0 Attached you will find dmesg.boot and pciconf -lvc. The kernel config is the 8.1-RELEASE(amd64) GENERIC with three added lines: device crypto options IPSEC options TCP_SIGNATURE These are for a TCP MD5 authenticated BGP session which doesn't go through either of the two affected interfaces. The kernel gave no messages when em1 stopped working. All other em cards (em0, em2, em3, em4, em5) seemed to be working fine. So, to recap: em1 stopped receiving packets (two nights ago, it was em0 with the exact same issue); doing "ifconfig em1 down;ifconfig em1 up" reboots the machine (didn't have console access at the time; don't know what exactly happened). The error happened at aroung 40kpps in, 40kpps out (counting only em1); two nights ago it was em0 who locked, and it was doing ~20kpps in each direction. After the em1 lockup em3 (working) went from 30 to 70kpps out (and 20->35 kpps in), no issues. As a side note, when having flowtable.enable=1, a large routing table change (at least 50000 changed routes) makes flowcleaner take 100% of one CPU. It stays there for hours even if I do flowtable.enable=0 in the meantime; the machine seems responsive though. Many thanks Alex --0-303701128-1284710967=:42781 Content-Type: text/plain; name="dmesg.boot.txt" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="dmesg.boot.txt" Q29weXJpZ2h0IChjKSAxOTkyLTIwMTAgVGhlIEZyZWVCU0QgUHJvamVjdC4K Q29weXJpZ2h0IChjKSAxOTc5LCAxOTgwLCAxOTgzLCAxOTg2LCAxOTg4LCAx OTg5LCAxOTkxLCAxOTkyLCAxOTkzLCAxOTk0CglUaGUgUmVnZW50cyBvZiB0 aGUgVW5pdmVyc2l0eSBvZiBDYWxpZm9ybmlhLiBBbGwgcmlnaHRzIHJlc2Vy dmVkLgpGcmVlQlNEIGlzIGEgcmVnaXN0ZXJlZCB0cmFkZW1hcmsgb2YgVGhl IEZyZWVCU0QgRm91bmRhdGlvbi4KRnJlZUJTRCA4LjEtUkVMRUFTRSAjMDog TW9uIFNlcCAxMyAxNDozMzowMiBFRVNUIDIwMTAKICAgIHJhem9yQGJvcmRl cjIuaW50ZXJjb25uZWN0LnJvOi91c3Ivb2JqL3Vzci9zcmMvc3lzL0JPUkRF UiBhbWQ2NApUaW1lY291bnRlciAiaTgyNTQiIGZyZXF1ZW5jeSAxMTkzMTgy IEh6IHF1YWxpdHkgMApDUFU6IEludGVsKFIpIFhlb24oUikgQ1BVICAgICAg ICAgICBFNTQyMCAgQCAyLjUwR0h6ICgyNTA2LjI0LU1IeiBLOC1jbGFzcyBD UFUpCiAgT3JpZ2luID0gIkdlbnVpbmVJbnRlbCIgIElkID0gMHgxMDY3NiAg RmFtaWx5ID0gNiAgTW9kZWwgPSAxNyAgU3RlcHBpbmcgPSA2CiAgRmVhdHVy ZXM9MHhiZmViZmJmZjxGUFUsVk1FLERFLFBTRSxUU0MsTVNSLFBBRSxNQ0Us Q1g4LEFQSUMsU0VQLE1UUlIsUEdFLE1DQSxDTU9WLFBBVCxQU0UzNixDTEZM VVNILERUUyxBQ1BJLE1NWCxGWFNSLFNTRSxTU0UyLFNTLEhUVCxUTSxQQkU+ CiAgRmVhdHVyZXMyPTB4Y2UzYmQ8U1NFMyxEVEVTNjQsTU9OLERTX0NQTCxW TVgsRVNULFRNMixTU1NFMyxDWDE2LHhUUFIsUERDTSxEQ0EsU1NFNC4xPgog IEFNRCBGZWF0dXJlcz0weDIwMTAwODAwPFNZU0NBTEwsTlgsTE0+CiAgQU1E IEZlYXR1cmVzMj0weDE8TEFIRj4KICBUU0M6IFAtc3RhdGUgaW52YXJpYW50 CnJlYWwgbWVtb3J5ICA9IDg1ODk5MzQ1OTIgKDgxOTIgTUIpCmF2YWlsIG1l bW9yeSA9IDgyMDU3NzQ4NDggKDc4MjUgTUIpCkFDUEkgQVBJQyBUYWJsZTog PElOVEVMICBTNTAwMFhWTj4KRnJlZUJTRC9TTVA6IE11bHRpcHJvY2Vzc29y IFN5c3RlbSBEZXRlY3RlZDogOCBDUFVzCkZyZWVCU0QvU01QOiAyIHBhY2th Z2UocykgeCA0IGNvcmUocykKIGNwdTAgKEJTUCk6IEFQSUMgSUQ6ICAzCiBj cHUxIChBUCk6IEFQSUMgSUQ6ICA0CiBjcHUyIChBUCk6IEFQSUMgSUQ6ICA1 CiBjcHUzIChBUCk6IEFQSUMgSUQ6ICA2CiBjcHU0IChBUCk6IEFQSUMgSUQ6 ICA3CiBjcHU1IChBUCk6IEFQSUMgSUQ6ICAwCiBjcHU2IChBUCk6IEFQSUMg SUQ6ICAxCiBjcHU3IChBUCk6IEFQSUMgSUQ6ICAyCmlvYXBpYzAgPFZlcnNp b24gMi4wPiBpcnFzIDAtMjMgb24gbW90aGVyYm9hcmQKaW9hcGljMSA8VmVy c2lvbiAyLjA+IGlycXMgMjQtNDcgb24gbW90aGVyYm9hcmQKbGFwaWMzOiBG b3JjaW5nIExJTlQxIHRvIGVkZ2UgdHJpZ2dlcgprYmQxIGF0IGtiZG11eDAK Y3J5cHRvc29mdDA6IDxzb2Z0d2FyZSBjcnlwdG8+IG9uIG1vdGhlcmJvYXJk CmFjcGkwOiA8SU5URUwgUzUwMDBYVk4+IG9uIG1vdGhlcmJvYXJkCmFjcGkw OiBbSVRIUkVBRF0KYWNwaTA6IFBvd2VyIEJ1dHRvbiAoZml4ZWQpCmFjcGkw OiByZXNlcnZhdGlvbiBvZiAwLCBhMDAwMCAoMykgZmFpbGVkClRpbWVjb3Vu dGVyICJBQ1BJLWZhc3QiIGZyZXF1ZW5jeSAzNTc5NTQ1IEh6IHF1YWxpdHkg MTAwMAphY3BpX3RpbWVyMDogPDI0LWJpdCB0aW1lciBhdCAzLjU3OTU0NU1I ej4gcG9ydCAweDQwOC0weDQwYiBvbiBhY3BpMApjcHUwOiA8QUNQSSBDUFU+ IG9uIGFjcGkwCmNwdTE6IDxBQ1BJIENQVT4gb24gYWNwaTAKY3B1MjogPEFD UEkgQ1BVPiBvbiBhY3BpMApjcHUzOiA8QUNQSSBDUFU+IG9uIGFjcGkwCmNw dTQ6IDxBQ1BJIENQVT4gb24gYWNwaTAKY3B1NTogPEFDUEkgQ1BVPiBvbiBh Y3BpMApjcHU2OiA8QUNQSSBDUFU+IG9uIGFjcGkwCmNwdTc6IDxBQ1BJIENQ VT4gb24gYWNwaTAKYWNwaV9ocGV0MDogPEhpZ2ggUHJlY2lzaW9uIEV2ZW50 IFRpbWVyPiBpb21lbSAweGZlZDAwMDAwLTB4ZmVkMDAzZmYgb24gYWNwaTAK VGltZWNvdW50ZXIgIkhQRVQiIGZyZXF1ZW5jeSAxNDMxODE4MCBIeiBxdWFs aXR5IDkwMAphY3BpX2J1dHRvbjA6IDxTbGVlcCBCdXR0b24+IG9uIGFjcGkw CmFjcGlfYnV0dG9uMTogPFBvd2VyIEJ1dHRvbj4gb24gYWNwaTAKcGNpYjA6 IDxBQ1BJIEhvc3QtUENJIGJyaWRnZT4gcG9ydCAweGNhMiwweGNhMywweGNm OC0weGNmZiBvbiBhY3BpMApwY2kwOiA8QUNQSSBQQ0kgYnVzPiBvbiBwY2li MApwY2liMTogPEFDUEkgUENJLVBDSSBicmlkZ2U+IGF0IGRldmljZSAyLjAg b24gcGNpMApwY2kxOiA8QUNQSSBQQ0kgYnVzPiBvbiBwY2liMQpwY2liMjog PEFDUEkgUENJLVBDSSBicmlkZ2U+IGlycSAxNiBhdCBkZXZpY2UgMC4wIG9u IHBjaTEKcGNpMjogPEFDUEkgUENJIGJ1cz4gb24gcGNpYjIKcGNpYjM6IDxB Q1BJIFBDSS1QQ0kgYnJpZGdlPiBpcnEgMTYgYXQgZGV2aWNlIDAuMCBvbiBw Y2kyCnBjaTM6IDxBQ1BJIFBDSSBidXM+IG9uIHBjaWIzCmVtMDogPEludGVs KFIpIFBSTy8xMDAwIE5ldHdvcmsgQ29ubmVjdGlvbiA3LjAuNT4gcG9ydCAw eDQwMjAtMHg0MDNmIG1lbSAweGMwOTYwMDAwLTB4YzA5N2ZmZmYsMHhjMDk0 MDAwMC0weGMwOTVmZmZmIGlycSAxNiBhdCBkZXZpY2UgMC4wIG9uIHBjaTMK ZW0wOiBVc2luZyBNU0kgaW50ZXJydXB0CmVtMDogW0ZJTFRFUl0KZW0wOiBF dGhlcm5ldCBhZGRyZXNzOiAwMDoxNToxNzoxNDo5NjoyYQplbTE6IDxJbnRl bChSKSBQUk8vMTAwMCBOZXR3b3JrIENvbm5lY3Rpb24gNy4wLjU+IHBvcnQg MHg0MDAwLTB4NDAxZiBtZW0gMHhjMDkyMDAwMC0weGMwOTNmZmZmLDB4YzA5 MDAwMDAtMHhjMDkxZmZmZiBpcnEgMTcgYXQgZGV2aWNlIDAuMSBvbiBwY2kz CmVtMTogVXNpbmcgTVNJIGludGVycnVwdAplbTE6IFtGSUxURVJdCmVtMTog RXRoZXJuZXQgYWRkcmVzczogMDA6MTU6MTc6MTQ6OTY6MmIKcGNpYjQ6IDxB Q1BJIFBDSS1QQ0kgYnJpZGdlPiBpcnEgMTggYXQgZGV2aWNlIDIuMCBvbiBw Y2kyCnBjaTQ6IDxBQ1BJIFBDSSBidXM+IG9uIHBjaWI0CmVtMjogPEludGVs KFIpIFBSTy8xMDAwIE5ldHdvcmsgQ29ubmVjdGlvbiA3LjAuNT4gcG9ydCAw eDMwMjAtMHgzMDNmIG1lbSAweGMwODIwMDAwLTB4YzA4M2ZmZmYsMHhjMDQw MDAwMC0weGMwN2ZmZmZmIGlycSAxOCBhdCBkZXZpY2UgMC4wIG9uIHBjaTQK ZW0yOiBVc2luZyBNU0kgaW50ZXJydXB0CmVtMjogW0ZJTFRFUl0KZW0yOiBF dGhlcm5ldCBhZGRyZXNzOiAwMDoxNToxNzo0YzpjMzoyOAplbTM6IDxJbnRl bChSKSBQUk8vMTAwMCBOZXR3b3JrIENvbm5lY3Rpb24gNy4wLjU+IHBvcnQg MHgzMDAwLTB4MzAxZiBtZW0gMHhjMDgwMDAwMC0weGMwODFmZmZmLDB4YzAw MDAwMDAtMHhjMDNmZmZmZiBpcnEgMTkgYXQgZGV2aWNlIDAuMSBvbiBwY2k0 CmVtMzogVXNpbmcgTVNJIGludGVycnVwdAplbTM6IFtGSUxURVJdCmVtMzog RXRoZXJuZXQgYWRkcmVzczogMDA6MTU6MTc6NGM6YzM6MjkKcGNpYjU6IDxB Q1BJIFBDSS1QQ0kgYnJpZGdlPiBhdCBkZXZpY2UgMC4zIG9uIHBjaTEKcGNp NTogPEFDUEkgUENJIGJ1cz4gb24gcGNpYjUKcGNpYjY6IDxQQ0ktUENJIGJy aWRnZT4gYXQgZGV2aWNlIDMuMCBvbiBwY2kwCnBjaTY6IDxQQ0kgYnVzPiBv biBwY2liNgpwY2liNzogPEFDUEkgUENJLVBDSSBicmlkZ2U+IGF0IGRldmlj ZSA0LjAgb24gcGNpMApwY2k3OiA8QUNQSSBQQ0kgYnVzPiBvbiBwY2liNwp2 Z2FwY2kwOiA8VkdBLWNvbXBhdGlibGUgZGlzcGxheT4gcG9ydCAweDIwMDAt MHgyMGZmIG1lbSAweGIwMDAwMDAwLTB4YmZmZmZmZmYsMHhjMGMwMDAwMC0w eGMwYzBmZmZmIGlycSAxNiBhdCBkZXZpY2UgMC4wIG9uIHBjaTcKcGNpNzog PG11bHRpbWVkaWEsIEhEQT4gYXQgZGV2aWNlIDAuMSAobm8gZHJpdmVyIGF0 dGFjaGVkKQpwY2liODogPEFDUEkgUENJLVBDSSBicmlkZ2U+IGF0IGRldmlj ZSA1LjAgb24gcGNpMApwY2k4OiA8QUNQSSBQQ0kgYnVzPiBvbiBwY2liOApw Y2liOTogPEFDUEkgUENJLVBDSSBicmlkZ2U+IGF0IGRldmljZSA2LjAgb24g cGNpMApwY2k5OiA8QUNQSSBQQ0kgYnVzPiBvbiBwY2liOQpwY2liMTA6IDxQ Q0ktUENJIGJyaWRnZT4gYXQgZGV2aWNlIDcuMCBvbiBwY2kwCnBjaTEwOiA8 UENJIGJ1cz4gb24gcGNpYjEwCnBjaTA6IDxtdWx0aW1lZGlhLCBIREE+IGF0 IGRldmljZSAyNy4wIChubyBkcml2ZXIgYXR0YWNoZWQpCnBjaWIxMTogPEFD UEkgUENJLVBDSSBicmlkZ2U+IGlycSAxNiBhdCBkZXZpY2UgMjguMCBvbiBw Y2kwCnBjaTExOiA8QUNQSSBQQ0kgYnVzPiBvbiBwY2liMTEKZW00OiA8SW50 ZWwoUikgUFJPLzEwMDAgTmV0d29yayBDb25uZWN0aW9uIDcuMC41PiBwb3J0 IDB4MTAyMC0weDEwM2YgbWVtIDB4YzBiNjAwMDAtMHhjMGI3ZmZmZiwweGMw YjQwMDAwLTB4YzBiNWZmZmYgaXJxIDE2IGF0IGRldmljZSAwLjAgb24gcGNp MTEKZW00OiBVc2luZyBNU0kgaW50ZXJydXB0CmVtNDogW0ZJTFRFUl0KZW00 OiBFdGhlcm5ldCBhZGRyZXNzOiAwMDoxNToxNzoyMToxNDoxNAplbTU6IDxJ bnRlbChSKSBQUk8vMTAwMCBOZXR3b3JrIENvbm5lY3Rpb24gNy4wLjU+IHBv cnQgMHgxMDAwLTB4MTAxZiBtZW0gMHhjMGIyMDAwMC0weGMwYjNmZmZmLDB4 YzBiMDAwMDAtMHhjMGIxZmZmZiBpcnEgMTcgYXQgZGV2aWNlIDAuMSBvbiBw Y2kxMQplbTU6IFVzaW5nIE1TSSBpbnRlcnJ1cHQKZW01OiBbRklMVEVSXQpl bTU6IEV0aGVybmV0IGFkZHJlc3M6IDAwOjE1OjE3OjIxOjE0OjE1CnVoY2kw OiA8SW50ZWwgNjMxWEVTQi82MzJYRVNCLzMxMDAgVVNCIGNvbnRyb2xsZXIg VVNCLTE+IHBvcnQgMHg1MDgwLTB4NTA5ZiBpcnEgMjMgYXQgZGV2aWNlIDI5 LjAgb24gcGNpMAp1aGNpMDogW0lUSFJFQURdCnVoY2kwOiBMZWdTdXAgPSAw eDJmMDAKdXNidXMwOiA8SW50ZWwgNjMxWEVTQi82MzJYRVNCLzMxMDAgVVNC IGNvbnRyb2xsZXIgVVNCLTE+IG9uIHVoY2kwCnVoY2kxOiA8SW50ZWwgNjMx WEVTQi82MzJYRVNCLzMxMDAgVVNCIGNvbnRyb2xsZXIgVVNCLTI+IHBvcnQg MHg1MDYwLTB4NTA3ZiBpcnEgMjIgYXQgZGV2aWNlIDI5LjEgb24gcGNpMAp1 aGNpMTogW0lUSFJFQURdCnVoY2kxOiBMZWdTdXAgPSAweDJmMDAKdXNidXMx OiA8SW50ZWwgNjMxWEVTQi82MzJYRVNCLzMxMDAgVVNCIGNvbnRyb2xsZXIg VVNCLTI+IG9uIHVoY2kxCnVoY2kyOiA8SW50ZWwgNjMxWEVTQi82MzJYRVNC LzMxMDAgVVNCIGNvbnRyb2xsZXIgVVNCLTM+IHBvcnQgMHg1MDQwLTB4NTA1 ZiBpcnEgMjMgYXQgZGV2aWNlIDI5LjIgb24gcGNpMAp1aGNpMjogW0lUSFJF QURdCnVoY2kyOiBMZWdTdXAgPSAweDJmMDAKdXNidXMyOiA8SW50ZWwgNjMx WEVTQi82MzJYRVNCLzMxMDAgVVNCIGNvbnRyb2xsZXIgVVNCLTM+IG9uIHVo Y2kyCnVoY2kzOiA8SW50ZWwgNjMxWEVTQi82MzJYRVNCLzMxMDAgVVNCIGNv bnRyb2xsZXIgVVNCLTQ+IHBvcnQgMHg1MDIwLTB4NTAzZiBpcnEgMjIgYXQg ZGV2aWNlIDI5LjMgb24gcGNpMAp1aGNpMzogW0lUSFJFQURdCnVoY2kzOiBM ZWdTdXAgPSAweDJmMDAKdXNidXMzOiA8SW50ZWwgNjMxWEVTQi82MzJYRVNC LzMxMDAgVVNCIGNvbnRyb2xsZXIgVVNCLTQ+IG9uIHVoY2kzCmVoY2kwOiA8 SW50ZWwgNjNYWEVTQiBVU0IgMi4wIGNvbnRyb2xsZXI+IG1lbSAweGMwZDA0 NDAwLTB4YzBkMDQ3ZmYgaXJxIDIzIGF0IGRldmljZSAyOS43IG9uIHBjaTAK ZWhjaTA6IFtJVEhSRUFEXQp1c2J1czQ6IEVIQ0kgdmVyc2lvbiAxLjAKdXNi dXM0OiA8SW50ZWwgNjNYWEVTQiBVU0IgMi4wIGNvbnRyb2xsZXI+IG9uIGVo Y2kwCnBjaWIxMjogPEFDUEkgUENJLVBDSSBicmlkZ2U+IGF0IGRldmljZSAz MC4wIG9uIHBjaTAKcGNpMTI6IDxBQ1BJIFBDSSBidXM+IG9uIHBjaWIxMgpp c2FiMDogPFBDSS1JU0EgYnJpZGdlPiBhdCBkZXZpY2UgMzEuMCBvbiBwY2kw CmlzYTA6IDxJU0EgYnVzPiBvbiBpc2FiMAphdGFwY2kwOiA8SW50ZWwgNjNY WEVTQjIgVURNQTEwMCBjb250cm9sbGVyPiBwb3J0IDB4MWYwLTB4MWY3LDB4 M2Y2LDB4MTcwLTB4MTc3LDB4Mzc2LDB4NTBiMC0weDUwYmYgaXJxIDIwIGF0 IGRldmljZSAzMS4xIG9uIHBjaTAKYXRhMDogPEFUQSBjaGFubmVsIDA+IG9u IGF0YXBjaTAKYXRhMDogW0lUSFJFQURdCmF0YXBjaTE6IDxJbnRlbCA2M1hY RVNCMiBTQVRBMzAwIGNvbnRyb2xsZXI+IHBvcnQgMHg1MGM4LTB4NTBjZiww eDUwZTQtMHg1MGU3LDB4NTBjMC0weDUwYzcsMHg1MGUwLTB4NTBlMywweDUw YTAtMHg1MGFmIG1lbSAweGMwZDA0MDAwLTB4YzBkMDQzZmYgaXJxIDIwIGF0 IGRldmljZSAzMS4yIG9uIHBjaTAKYXRhcGNpMTogW0lUSFJFQURdCmF0YTI6 IDxBVEEgY2hhbm5lbCAwPiBvbiBhdGFwY2kxCmF0YTI6IFtJVEhSRUFEXQph dGEzOiA8QVRBIGNoYW5uZWwgMT4gb24gYXRhcGNpMQphdGEzOiBbSVRIUkVB RF0KcGNpMDogPHNlcmlhbCBidXMsIFNNQnVzPiBhdCBkZXZpY2UgMzEuMyAo bm8gZHJpdmVyIGF0dGFjaGVkKQphdHJ0YzA6IDxBVCByZWFsdGltZSBjbG9j az4gcG9ydCAweDcwLTB4NzEsMHg3NC0weDc3IGlycSA4IG9uIGFjcGkwCmF0 a2JkYzA6IDxLZXlib2FyZCBjb250cm9sbGVyIChpODA0Mik+IHBvcnQgMHg2 MCwweDY0IGlycSAxIG9uIGFjcGkwCmF0a2JkMDogPEFUIEtleWJvYXJkPiBp cnEgMSBvbiBhdGtiZGMwCmtiZDAgYXQgYXRrYmQwCmF0a2JkMDogW0dJQU5U LUxPQ0tFRF0KYXRrYmQwOiBbSVRIUkVBRF0KcHNtMDogPFBTLzIgTW91c2U+ IGlycSAxMiBvbiBhdGtiZGMwCnBzbTA6IFtHSUFOVC1MT0NLRURdCnBzbTA6 IFtJVEhSRUFEXQpwc20wOiBtb2RlbCBJbnRlbGxpTW91c2UgRXhwbG9yZXIs IGRldmljZSBJRCA0CnVhcnQwOiA8MTY1NTAgb3IgY29tcGF0aWJsZT4gcG9y dCAweDNmOC0weDNmZiBpcnEgNCBmbGFncyAweDEwIG9uIGFjcGkwCnVhcnQw OiBbRklMVEVSXQp1YXJ0MTogPDE2NTUwIG9yIGNvbXBhdGlibGU+IHBvcnQg MHgyZjgtMHgyZmYgaXJxIDMgb24gYWNwaTAKdWFydDE6IFtGSUxURVJdCnNj MDogPFN5c3RlbSBjb25zb2xlPiBhdCBmbGFncyAweDEwMCBvbiBpc2EwCnNj MDogVkdBIDwxNiB2aXJ0dWFsIGNvbnNvbGVzLCBmbGFncz0weDMwMD4Kdmdh MDogPEdlbmVyaWMgSVNBIFZHQT4gYXQgcG9ydCAweDNjMC0weDNkZiBpb21l bSAweGEwMDAwLTB4YmZmZmYgb24gaXNhMApwcGMwOiBjYW5ub3QgcmVzZXJ2 ZSBJL08gcG9ydCByYW5nZQplc3QwOiA8RW5oYW5jZWQgU3BlZWRTdGVwIEZy ZXF1ZW5jeSBDb250cm9sPiBvbiBjcHUwCnA0dGNjMDogPENQVSBGcmVxdWVu Y3kgVGhlcm1hbCBDb250cm9sPiBvbiBjcHUwCmVzdDE6IDxFbmhhbmNlZCBT cGVlZFN0ZXAgRnJlcXVlbmN5IENvbnRyb2w+IG9uIGNwdTEKcDR0Y2MxOiA8 Q1BVIEZyZXF1ZW5jeSBUaGVybWFsIENvbnRyb2w+IG9uIGNwdTEKZXN0Mjog PEVuaGFuY2VkIFNwZWVkU3RlcCBGcmVxdWVuY3kgQ29udHJvbD4gb24gY3B1 MgpwNHRjYzI6IDxDUFUgRnJlcXVlbmN5IFRoZXJtYWwgQ29udHJvbD4gb24g Y3B1Mgplc3QzOiA8RW5oYW5jZWQgU3BlZWRTdGVwIEZyZXF1ZW5jeSBDb250 cm9sPiBvbiBjcHUzCnA0dGNjMzogPENQVSBGcmVxdWVuY3kgVGhlcm1hbCBD b250cm9sPiBvbiBjcHUzCmVzdDQ6IDxFbmhhbmNlZCBTcGVlZFN0ZXAgRnJl cXVlbmN5IENvbnRyb2w+IG9uIGNwdTQKcDR0Y2M0OiA8Q1BVIEZyZXF1ZW5j eSBUaGVybWFsIENvbnRyb2w+IG9uIGNwdTQKZXN0NTogPEVuaGFuY2VkIFNw ZWVkU3RlcCBGcmVxdWVuY3kgQ29udHJvbD4gb24gY3B1NQpwNHRjYzU6IDxD UFUgRnJlcXVlbmN5IFRoZXJtYWwgQ29udHJvbD4gb24gY3B1NQplc3Q2OiA8 RW5oYW5jZWQgU3BlZWRTdGVwIEZyZXF1ZW5jeSBDb250cm9sPiBvbiBjcHU2 CnA0dGNjNjogPENQVSBGcmVxdWVuY3kgVGhlcm1hbCBDb250cm9sPiBvbiBj cHU2CmVzdDc6IDxFbmhhbmNlZCBTcGVlZFN0ZXAgRnJlcXVlbmN5IENvbnRy b2w+IG9uIGNwdTcKcDR0Y2M3OiA8Q1BVIEZyZXF1ZW5jeSBUaGVybWFsIENv bnRyb2w+IG9uIGNwdTcKUlRDIEJJT1MgZGlhZ25vc3RpYyBlcnJvciBlPGZp eGVkX2Rpc2ssaW52YWxpZF90aW1lPgpUaW1lY291bnRlcnMgdGljayBldmVy eSAxLjAwMCBtc2VjCklQc2VjOiBJbml0aWFsaXplZCBTZWN1cml0eSBBc3Nv Y2lhdGlvbiBQcm9jZXNzaW5nLgp1c2J1czA6IDEyTWJwcyBGdWxsIFNwZWVk IFVTQiB2MS4wCnVzYnVzMTogMTJNYnBzIEZ1bGwgU3BlZWQgVVNCIHYxLjAK dXNidXMyOiAxMk1icHMgRnVsbCBTcGVlZCBVU0IgdjEuMAp1c2J1czM6IDEy TWJwcyBGdWxsIFNwZWVkIFVTQiB2MS4wCnVzYnVzNDogNDgwTWJwcyBIaWdo IFNwZWVkIFVTQiB2Mi4wCnVnZW4wLjE6IDxJbnRlbD4gYXQgdXNidXMwCnVo dWIwOiA8SW50ZWwgVUhDSSByb290IEhVQiwgY2xhc3MgOS8wLCByZXYgMS4w MC8xLjAwLCBhZGRyIDE+IG9uIHVzYnVzMAp1Z2VuMS4xOiA8SW50ZWw+IGF0 IHVzYnVzMQp1aHViMTogPEludGVsIFVIQ0kgcm9vdCBIVUIsIGNsYXNzIDkv MCwgcmV2IDEuMDAvMS4wMCwgYWRkciAxPiBvbiB1c2J1czEKdWdlbjIuMTog PEludGVsPiBhdCB1c2J1czIKdWh1YjI6IDxJbnRlbCBVSENJIHJvb3QgSFVC LCBjbGFzcyA5LzAsIHJldiAxLjAwLzEuMDAsIGFkZHIgMT4gb24gdXNidXMy CnVnZW4zLjE6IDxJbnRlbD4gYXQgdXNidXMzCnVodWIzOiA8SW50ZWwgVUhD SSByb290IEhVQiwgY2xhc3MgOS8wLCByZXYgMS4wMC8xLjAwLCBhZGRyIDE+ IG9uIHVzYnVzMwp1Z2VuNC4xOiA8SW50ZWw+IGF0IHVzYnVzNAp1aHViNDog PEludGVsIEVIQ0kgcm9vdCBIVUIsIGNsYXNzIDkvMCwgcmV2IDIuMDAvMS4w MCwgYWRkciAxPiBvbiB1c2J1czQKYWNkMDogRFZEUiA8VFNTVGNvcnBDRC9E VkRXIFNILVMxODJEL1NCMDQ+IGF0IGF0YTAtc2xhdmUgVURNQTMzIAphZDQ6 IDcwOTExTUIgPFdEQyBXRDc0MEFERkQtMDBOTFI1IDIxLjA3UVI1PiBhdCBh dGEyLW1hc3RlciBVRE1BMTAwIFNBVEEKYWQ0OiBVbnVzYWJsZSBWaXJ0dWFs IERpc2sKbGFwaWMxOiBGb3JjaW5nIExJTlQxIHRvIGVkZ2UgdHJpZ2dlcgpT TVA6IEFQIENQVSAjNiBMYXVuY2hlZCEKbGFwaWMyOiBGb3JjaW5nIExJTlQx IHRvIGVkZ2UgdHJpZ2dlcgpTTVA6IEFQIENQVSAjNyBMYXVuY2hlZCEKbGFw aWM2OiBGb3JjaW5nIExJTlQxIHRvIGVkZ2UgdHJpZ2dlcgpTTVA6IEFQIENQ VSAjMyBMYXVuY2hlZCEKbGFwaWMwOiBGb3JjaW5nIExJTlQxIHRvIGVkZ2Ug dHJpZ2dlcgpTTVA6IEFQIENQVSAjNSBMYXVuY2hlZCEKbGFwaWM3OiBGb3Jj aW5nIExJTlQxIHRvIGVkZ2UgdHJpZ2dlcgpTTVA6IEFQIENQVSAjNCBMYXVu Y2hlZCEKbGFwaWM1OiBGb3JjaW5nIExJTlQxIHRvIGVkZ2UgdHJpZ2dlcgpT TVA6IEFQIENQVSAjMiBMYXVuY2hlZCEKbGFwaWM0OiBGb3JjaW5nIExJTlQx IHRvIGVkZ2UgdHJpZ2dlcgpTTVA6IEFQIENQVSAjMSBMYXVuY2hlZCEKUm9v dCBtb3VudCB3YWl0aW5nIGZvcjogdXNidXM0IHVzYnVzMyB1c2J1czIgdXNi dXMxIHVzYnVzMAp1aHViMDogMiBwb3J0cyB3aXRoIDIgcmVtb3ZhYmxlLCBz ZWxmIHBvd2VyZWQKdWh1YjI6IDIgcG9ydHMgd2l0aCAyIHJlbW92YWJsZSwg c2VsZiBwb3dlcmVkCnVodWIxOiAyIHBvcnRzIHdpdGggMiByZW1vdmFibGUs IHNlbGYgcG93ZXJlZAp1aHViMzogMiBwb3J0cyB3aXRoIDIgcmVtb3ZhYmxl LCBzZWxmIHBvd2VyZWQKUm9vdCBtb3VudCB3YWl0aW5nIGZvcjogdXNidXM0 ClJvb3QgbW91bnQgd2FpdGluZyBmb3I6IHVzYnVzNApSb290IG1vdW50IHdh aXRpbmcgZm9yOiB1c2J1czQKdWh1YjQ6IDggcG9ydHMgd2l0aCA4IHJlbW92 YWJsZSwgc2VsZiBwb3dlcmVkClRyeWluZyB0byBtb3VudCByb290IGZyb20g dWZzOi9kZXYvYWQ0czFhCldBUk5JTkc6IC8gd2FzIG5vdCBwcm9wZXJseSBk aXNtb3VudGVkCldBUk5JTkc6IC90bXAgd2FzIG5vdCBwcm9wZXJseSBkaXNt b3VudGVkCldBUk5JTkc6IC91c3Igd2FzIG5vdCBwcm9wZXJseSBkaXNtb3Vu dGVkCldBUk5JTkc6IC92YXIgd2FzIG5vdCBwcm9wZXJseSBkaXNtb3VudGVk Cg== --0-303701128-1284710967=:42781 Content-Type: text/plain; name="pciconflvc.txt" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="pciconflvc.txt" aG9zdGIwQHBjaTA6MDowOjA6CWNsYXNzPTB4MDYwMDAwIGNhcmQ9MHgzNDcy ODA4NiBjaGlwPTB4MjVjMDgwODYgcmV2PTB4MzEgaGRyPTB4MDAKICAgIHZl bmRvciAgICAgPSAnSW50ZWwgQ29ycG9yYXRpb24nCiAgICBkZXZpY2UgICAg ID0gJzUwMDBYIENoaXBzZXQgTWVtb3J5IENvbnRyb2xsZXIgSHViJwogICAg Y2xhc3MgICAgICA9IGJyaWRnZQogICAgc3ViY2xhc3MgICA9IEhPU1QtUENJ CiAgICBjYXAgMDFbNTBdID0gcG93ZXJzcGVjIDIgIHN1cHBvcnRzIEQwIEQz ICBjdXJyZW50IEQwCiAgICBjYXAgMDVbNThdID0gTVNJIHN1cHBvcnRzIDIg bWVzc2FnZXMgCiAgICBjYXAgMTBbNmNdID0gUENJLUV4cHJlc3MgMSByb290 IHBvcnQgbWF4IGRhdGEgMTI4KDI1NikgbGluayB4NCh4NCkKcGNpYjFAcGNp MDowOjI6MDoJY2xhc3M9MHgwNjA0MDAgY2FyZD0weDAwMDAwMDAwIGNoaXA9 MHgyNWY3ODA4NiByZXY9MHgzMSBoZHI9MHgwMQogICAgdmVuZG9yICAgICA9 ICdJbnRlbCBDb3Jwb3JhdGlvbicKICAgIGRldmljZSAgICAgPSAnNTAwMCBT ZXJpZXMgQ2hpcHNldCBQQ0llIHg4IFBvcnQgMi0zJwogICAgY2xhc3MgICAg ICA9IGJyaWRnZQogICAgc3ViY2xhc3MgICA9IFBDSS1QQ0kKICAgIGNhcCAw MVs1MF0gPSBwb3dlcnNwZWMgMiAgc3VwcG9ydHMgRDAgRDMgIGN1cnJlbnQg RDAKICAgIGNhcCAwNVs1OF0gPSBNU0kgc3VwcG9ydHMgMiBtZXNzYWdlcyAK ICAgIGNhcCAxMFs2Y10gPSBQQ0ktRXhwcmVzcyAxIHJvb3QgcG9ydCBtYXgg ZGF0YSAxMjgoMjU2KSBsaW5rIHg4KHg4KQpwY2liNkBwY2kwOjA6MzowOglj bGFzcz0weDA2MDQwMCBjYXJkPTB4MDAwMDAwMDAgY2hpcD0weDI1ZTM4MDg2 IHJldj0weDMxIGhkcj0weDAxCiAgICB2ZW5kb3IgICAgID0gJ0ludGVsIENv cnBvcmF0aW9uJwogICAgZGV2aWNlICAgICA9ICc1MDAwIFNlcmllcyBDaGlw c2V0IFBDSWUgeDQgUG9ydCAzJwogICAgY2xhc3MgICAgICA9IGJyaWRnZQog ICAgc3ViY2xhc3MgICA9IFBDSS1QQ0kKICAgIGNhcCAwMVs1MF0gPSBwb3dl cnNwZWMgMiAgc3VwcG9ydHMgRDAgRDMgIGN1cnJlbnQgRDAKICAgIGNhcCAw NVs1OF0gPSBNU0kgc3VwcG9ydHMgMiBtZXNzYWdlcyAKICAgIGNhcCAxMFs2 Y10gPSBQQ0ktRXhwcmVzcyAxIHJvb3QgcG9ydCBtYXggZGF0YSAxMjgoMjU2 KSBsaW5rIHgwKHg0KQpwY2liN0BwY2kwOjA6NDowOgljbGFzcz0weDA2MDQw MCBjYXJkPTB4MDAwMDAwMDAgY2hpcD0weDI1ZmE4MDg2IHJldj0weDMxIGhk cj0weDAxCiAgICB2ZW5kb3IgICAgID0gJ0ludGVsIENvcnBvcmF0aW9uJwog ICAgZGV2aWNlICAgICA9ICc1MDAwWCBDaGlwc2V0IFBDSWUgeDE2IFBvcnQg NC03JwogICAgY2xhc3MgICAgICA9IGJyaWRnZQogICAgc3ViY2xhc3MgICA9 IFBDSS1QQ0kKICAgIGNhcCAwMVs1MF0gPSBwb3dlcnNwZWMgMiAgc3VwcG9y dHMgRDAgRDMgIGN1cnJlbnQgRDAKICAgIGNhcCAwNVs1OF0gPSBNU0kgc3Vw cG9ydHMgMiBtZXNzYWdlcyAKICAgIGNhcCAxMFs2Y10gPSBQQ0ktRXhwcmVz cyAxIHJvb3QgcG9ydCBtYXggZGF0YSAxMjgoMjU2KSBsaW5rIHgxNih4MTYp CnBjaWI4QHBjaTA6MDo1OjA6CWNsYXNzPTB4MDYwNDAwIGNhcmQ9MHgwMDAw MDAwMCBjaGlwPTB4MjVlNTgwODYgcmV2PTB4MzEgaGRyPTB4MDEKICAgIHZl bmRvciAgICAgPSAnSW50ZWwgQ29ycG9yYXRpb24nCiAgICBkZXZpY2UgICAg ID0gJzUwMDAgU2VyaWVzIENoaXBzZXQgUENJZSB4NCBQb3J0IDUnCiAgICBj bGFzcyAgICAgID0gYnJpZGdlCiAgICBzdWJjbGFzcyAgID0gUENJLVBDSQog ICAgY2FwIDAxWzUwXSA9IHBvd2Vyc3BlYyAyICBzdXBwb3J0cyBEMCBEMyAg Y3VycmVudCBEMAogICAgY2FwIDA1WzU4XSA9IE1TSSBzdXBwb3J0cyAyIG1l c3NhZ2VzIAogICAgY2FwIDEwWzZjXSA9IFBDSS1FeHByZXNzIDEgcm9vdCBw b3J0IG1heCBkYXRhIDEyOCgyNTYpIGxpbmsgeDAoeDQpCnBjaWI5QHBjaTA6 MDo2OjA6CWNsYXNzPTB4MDYwNDAwIGNhcmQ9MHgwMDAwMDAwMCBjaGlwPTB4 MjVlNjgwODYgcmV2PTB4MzEgaGRyPTB4MDEKICAgIHZlbmRvciAgICAgPSAn SW50ZWwgQ29ycG9yYXRpb24nCiAgICBkZXZpY2UgICAgID0gJzUwMDAgU2Vy aWVzIENoaXBzZXQgUENJZSB4NCBQb3J0IDYnCiAgICBjbGFzcyAgICAgID0g YnJpZGdlCiAgICBzdWJjbGFzcyAgID0gUENJLVBDSQogICAgY2FwIDAxWzUw XSA9IHBvd2Vyc3BlYyAyICBzdXBwb3J0cyBEMCBEMyAgY3VycmVudCBEMAog ICAgY2FwIDA1WzU4XSA9IE1TSSBzdXBwb3J0cyAyIG1lc3NhZ2VzIAogICAg Y2FwIDEwWzZjXSA9IFBDSS1FeHByZXNzIDEgcm9vdCBwb3J0IG1heCBkYXRh IDEyOCgyNTYpIGxpbmsgeDAoeDQpCnBjaWIxMEBwY2kwOjA6NzowOgljbGFz cz0weDA2MDQwMCBjYXJkPTB4MDAwMDAwMDAgY2hpcD0weDI1ZTc4MDg2IHJl dj0weDMxIGhkcj0weDAxCiAgICB2ZW5kb3IgICAgID0gJ0ludGVsIENvcnBv cmF0aW9uJwogICAgZGV2aWNlICAgICA9ICc1MDAwIFNlcmllcyBDaGlwc2V0 IFBDSWUgeDQgUG9ydCA3JwogICAgY2xhc3MgICAgICA9IGJyaWRnZQogICAg c3ViY2xhc3MgICA9IFBDSS1QQ0kKICAgIGNhcCAwMVs1MF0gPSBwb3dlcnNw ZWMgMiAgc3VwcG9ydHMgRDAgRDMgIGN1cnJlbnQgRDAKICAgIGNhcCAwNVs1 OF0gPSBNU0kgc3VwcG9ydHMgMiBtZXNzYWdlcyAKICAgIGNhcCAxMFs2Y10g PSBQQ0ktRXhwcmVzcyAxIHJvb3QgcG9ydCBtYXggZGF0YSAxMjgoMjU2KSBs aW5rIHgwKHg0KQpob3N0YjFAcGNpMDowOjE2OjA6CWNsYXNzPTB4MDYwMDAw IGNhcmQ9MHgzNDcyODA4NiBjaGlwPTB4MjVmMDgwODYgcmV2PTB4MzEgaGRy PTB4MDAKICAgIHZlbmRvciAgICAgPSAnSW50ZWwgQ29ycG9yYXRpb24nCiAg ICBkZXZpY2UgICAgID0gJzUwMDAgU2VyaWVzIENoaXBzZXQgRXJyb3IgUmVw b3J0aW5nIFJlZ2lzdGVycycKICAgIGNsYXNzICAgICAgPSBicmlkZ2UKICAg IHN1YmNsYXNzICAgPSBIT1NULVBDSQpob3N0YjJAcGNpMDowOjE2OjE6CWNs YXNzPTB4MDYwMDAwIGNhcmQ9MHgzNDcyODA4NiBjaGlwPTB4MjVmMDgwODYg cmV2PTB4MzEgaGRyPTB4MDAKICAgIHZlbmRvciAgICAgPSAnSW50ZWwgQ29y cG9yYXRpb24nCiAgICBkZXZpY2UgICAgID0gJzUwMDAgU2VyaWVzIENoaXBz ZXQgRXJyb3IgUmVwb3J0aW5nIFJlZ2lzdGVycycKICAgIGNsYXNzICAgICAg PSBicmlkZ2UKICAgIHN1YmNsYXNzICAgPSBIT1NULVBDSQpob3N0YjNAcGNp MDowOjE2OjI6CWNsYXNzPTB4MDYwMDAwIGNhcmQ9MHgzNDcyODA4NiBjaGlw PTB4MjVmMDgwODYgcmV2PTB4MzEgaGRyPTB4MDAKICAgIHZlbmRvciAgICAg PSAnSW50ZWwgQ29ycG9yYXRpb24nCiAgICBkZXZpY2UgICAgID0gJzUwMDAg U2VyaWVzIENoaXBzZXQgRXJyb3IgUmVwb3J0aW5nIFJlZ2lzdGVycycKICAg IGNsYXNzICAgICAgPSBicmlkZ2UKICAgIHN1YmNsYXNzICAgPSBIT1NULVBD SQpob3N0YjRAcGNpMDowOjE3OjA6CWNsYXNzPTB4MDYwMDAwIGNhcmQ9MHgz NDcyODA4NiBjaGlwPTB4MjVmMTgwODYgcmV2PTB4MzEgaGRyPTB4MDAKICAg IHZlbmRvciAgICAgPSAnSW50ZWwgQ29ycG9yYXRpb24nCiAgICBkZXZpY2Ug ICAgID0gJzUwMDAgU2VyaWVzIENoaXBzZXQgUmVzZXJ2ZWQgUmVnaXN0ZXJz JwogICAgY2xhc3MgICAgICA9IGJyaWRnZQogICAgc3ViY2xhc3MgICA9IEhP U1QtUENJCmhvc3RiNUBwY2kwOjA6MTk6MDoJY2xhc3M9MHgwNjAwMDAgY2Fy ZD0weDM0NzI4MDg2IGNoaXA9MHgyNWYzODA4NiByZXY9MHgzMSBoZHI9MHgw MAogICAgdmVuZG9yICAgICA9ICdJbnRlbCBDb3Jwb3JhdGlvbicKICAgIGRl dmljZSAgICAgPSAnNTAwMCBTZXJpZXMgQ2hpcHNldCBSZXNlcnZlZCBSZWdp c3RlcnMnCiAgICBjbGFzcyAgICAgID0gYnJpZGdlCiAgICBzdWJjbGFzcyAg ID0gSE9TVC1QQ0kKaG9zdGI2QHBjaTA6MDoyMTowOgljbGFzcz0weDA2MDAw MCBjYXJkPTB4MzQ3MjgwODYgY2hpcD0weDI1ZjU4MDg2IHJldj0weDMxIGhk cj0weDAwCiAgICB2ZW5kb3IgICAgID0gJ0ludGVsIENvcnBvcmF0aW9uJwog ICAgZGV2aWNlICAgICA9ICc1MDAwIFNlcmllcyBDaGlwc2V0IEZCRCBSZWdp c3RlcnMnCiAgICBjbGFzcyAgICAgID0gYnJpZGdlCiAgICBzdWJjbGFzcyAg ID0gSE9TVC1QQ0kKaG9zdGI3QHBjaTA6MDoyMjowOgljbGFzcz0weDA2MDAw MCBjYXJkPTB4MzQ3MjgwODYgY2hpcD0weDI1ZjY4MDg2IHJldj0weDMxIGhk cj0weDAwCiAgICB2ZW5kb3IgICAgID0gJ0ludGVsIENvcnBvcmF0aW9uJwog ICAgZGV2aWNlICAgICA9ICc1MDAwIFNlcmllcyBDaGlwc2V0IEZCRCBSZWdp c3RlcnMnCiAgICBjbGFzcyAgICAgID0gYnJpZGdlCiAgICBzdWJjbGFzcyAg ID0gSE9TVC1QQ0kKbm9uZTBAcGNpMDowOjI3OjA6CWNsYXNzPTB4MDQwMzAw IGNhcmQ9MHgzNDcyODA4NiBjaGlwPTB4MjY5YTgwODYgcmV2PTB4MDkgaGRy PTB4MDAKICAgIHZlbmRvciAgICAgPSAnSW50ZWwgQ29ycG9yYXRpb24nCiAg ICBkZXZpY2UgICAgID0gJ0VudGVycHJpc2UgU291dGhicmlkZ2UgSGlnaCBE ZWZpbml0aW9uIEF1ZGlvJwogICAgY2xhc3MgICAgICA9IG11bHRpbWVkaWEK ICAgIHN1YmNsYXNzICAgPSBIREEKICAgIGNhcCAwMVs1MF0gPSBwb3dlcnNw ZWMgMiAgc3VwcG9ydHMgRDAgRDMgIGN1cnJlbnQgRDAKICAgIGNhcCAwNVs2 MF0gPSBNU0kgc3VwcG9ydHMgMSBtZXNzYWdlLCA2NCBiaXQgCiAgICBjYXAg MTBbNzBdID0gUENJLUV4cHJlc3MgMSByb290IGVuZHBvaW50IG1heCBkYXRh IDEyOCgxMjgpIGxpbmsgeDAoeDApCnBjaWIxMUBwY2kwOjA6Mjg6MDoJY2xh c3M9MHgwNjA0MDAgY2FyZD0weDM0NzI4MDg2IGNoaXA9MHgyNjkwODA4NiBy ZXY9MHgwOSBoZHI9MHgwMQogICAgdmVuZG9yICAgICA9ICdJbnRlbCBDb3Jw b3JhdGlvbicKICAgIGRldmljZSAgICAgPSAnNjMxeEVTQi82MzJ4RVNCLzMx MDAgUENJZSBSb290IFBvcnQgMScKICAgIGNsYXNzICAgICAgPSBicmlkZ2UK ICAgIHN1YmNsYXNzICAgPSBQQ0ktUENJCiAgICBjYXAgMTBbNDBdID0gUENJ LUV4cHJlc3MgMSByb290IHBvcnQgbWF4IGRhdGEgMTI4KDEyOCkgbGluayB4 NCh4NCkKICAgIGNhcCAwNVs4MF0gPSBNU0kgc3VwcG9ydHMgMSBtZXNzYWdl IAogICAgY2FwIDBkWzkwXSA9IFBDSSBCcmlkZ2UgY2FyZD0weDM0NzI4MDg2 CiAgICBjYXAgMDFbYTBdID0gcG93ZXJzcGVjIDIgIHN1cHBvcnRzIEQwIEQz ICBjdXJyZW50IEQwCnVoY2kwQHBjaTA6MDoyOTowOgljbGFzcz0weDBjMDMw MCBjYXJkPTB4MzQ3MjgwODYgY2hpcD0weDI2ODg4MDg2IHJldj0weDA5IGhk cj0weDAwCiAgICB2ZW5kb3IgICAgID0gJ0ludGVsIENvcnBvcmF0aW9uJwog ICAgZGV2aWNlICAgICA9ICc2MzF4RVNCLzYzMnhFU0IvMzEwMCBDaGlwc2V0 IFVTQiBVbml2ZXJzYWwgSG9zdCBDb250cm9sbGVyICoxJwogICAgY2xhc3Mg ICAgICA9IHNlcmlhbCBidXMKICAgIHN1YmNsYXNzICAgPSBVU0IKdWhjaTFA cGNpMDowOjI5OjE6CWNsYXNzPTB4MGMwMzAwIGNhcmQ9MHgzNDcyODA4NiBj aGlwPTB4MjY4OTgwODYgcmV2PTB4MDkgaGRyPTB4MDAKICAgIHZlbmRvciAg ICAgPSAnSW50ZWwgQ29ycG9yYXRpb24nCiAgICBkZXZpY2UgICAgID0gJzYz MXhFU0IvNjMyeEVTQi8zMTAwIENoaXBzZXQgVVNCIFVuaXZlcnNhbCBIb3N0 IENvbnRyb2xsZXIgKjInCiAgICBjbGFzcyAgICAgID0gc2VyaWFsIGJ1cwog ICAgc3ViY2xhc3MgICA9IFVTQgp1aGNpMkBwY2kwOjA6Mjk6MjoJY2xhc3M9 MHgwYzAzMDAgY2FyZD0weDM0NzI4MDg2IGNoaXA9MHgyNjhhODA4NiByZXY9 MHgwOSBoZHI9MHgwMAogICAgdmVuZG9yICAgICA9ICdJbnRlbCBDb3Jwb3Jh dGlvbicKICAgIGRldmljZSAgICAgPSAnNjMxeEVTQi82MzJ4RVNCLzMxMDAg Q2hpcHNldCBVU0IgVW5pdmVyc2FsIEhvc3QgQ29udHJvbGxlciAqMycKICAg IGNsYXNzICAgICAgPSBzZXJpYWwgYnVzCiAgICBzdWJjbGFzcyAgID0gVVNC CnVoY2kzQHBjaTA6MDoyOTozOgljbGFzcz0weDBjMDMwMCBjYXJkPTB4MzQ3 MjgwODYgY2hpcD0weDI2OGI4MDg2IHJldj0weDA5IGhkcj0weDAwCiAgICB2 ZW5kb3IgICAgID0gJ0ludGVsIENvcnBvcmF0aW9uJwogICAgZGV2aWNlICAg ICA9ICc2MzF4RVNCLzYzMnhFU0IvMzEwMCBDaGlwc2V0IFVTQiBVbml2ZXJz YWwgSG9zdCBDb250cm9sbGVyICo0JwogICAgY2xhc3MgICAgICA9IHNlcmlh bCBidXMKICAgIHN1YmNsYXNzICAgPSBVU0IKZWhjaTBAcGNpMDowOjI5Ojc6 CWNsYXNzPTB4MGMwMzIwIGNhcmQ9MHgzNDcyODA4NiBjaGlwPTB4MjY4Yzgw ODYgcmV2PTB4MDkgaGRyPTB4MDAKICAgIHZlbmRvciAgICAgPSAnSW50ZWwg Q29ycG9yYXRpb24nCiAgICBkZXZpY2UgICAgID0gJzYzMXhFU0IvNjMyeEVT Qi8zMTAwIENoaXBzZXQgVVNCMiBFbmhhbmNlZCBIb3N0IENvbnRyb2xsZXIn CiAgICBjbGFzcyAgICAgID0gc2VyaWFsIGJ1cwogICAgc3ViY2xhc3MgICA9 IFVTQgogICAgY2FwIDAxWzUwXSA9IHBvd2Vyc3BlYyAyICBzdXBwb3J0cyBE MCBEMyAgY3VycmVudCBEMAogICAgY2FwIDBhWzU4XSA9IEVIQ0kgRGVidWcg UG9ydCBhdCBvZmZzZXQgMHhhMCBpbiBtYXAgMHgxNApwY2liMTJAcGNpMDow OjMwOjA6CWNsYXNzPTB4MDYwNDAxIGNhcmQ9MHgzNDcyODA4NiBjaGlwPTB4 MjQ0ZTgwODYgcmV2PTB4ZDkgaGRyPTB4MDEKICAgIHZlbmRvciAgICAgPSAn SW50ZWwgQ29ycG9yYXRpb24nCiAgICBkZXZpY2UgICAgID0gJzgyODAxIEZh bWlseSAoSUNIMi8zLzQvNS82LzcvOC85LDYzeHhFU0IpIEh1YiBJbnRlcmZh Y2UgdG8gUENJIEJyaWRnZScKICAgIGNsYXNzICAgICAgPSBicmlkZ2UKICAg IHN1YmNsYXNzICAgPSBQQ0ktUENJCiAgICBjYXAgMGRbNTBdID0gUENJIEJy aWRnZSBjYXJkPTB4MzQ3MjgwODYKaXNhYjBAcGNpMDowOjMxOjA6CWNsYXNz PTB4MDYwMTAwIGNhcmQ9MHgzNDcyODA4NiBjaGlwPTB4MjY3MDgwODYgcmV2 PTB4MDkgaGRyPTB4MDAKICAgIHZlbmRvciAgICAgPSAnSW50ZWwgQ29ycG9y YXRpb24nCiAgICBkZXZpY2UgICAgID0gJ0xQQyBJbnRlcmZhY2UgQ29udHJv bGxlciAoNjMxeEVTQi82MzIxRVNCLzMxMDAgKScKICAgIGNsYXNzICAgICAg PSBicmlkZ2UKICAgIHN1YmNsYXNzICAgPSBQQ0ktSVNBCmF0YXBjaTBAcGNp MDowOjMxOjE6CWNsYXNzPTB4MDEwMThhIGNhcmQ9MHgzNDcyODA4NiBjaGlw PTB4MjY5ZTgwODYgcmV2PTB4MDkgaGRyPTB4MDAKICAgIHZlbmRvciAgICAg PSAnSW50ZWwgQ29ycG9yYXRpb24nCiAgICBkZXZpY2UgICAgID0gJzYzMXhF U0IvNjMyeEVTQi8zMTAwIFVsdHJhIEFUQSBTdG9yYWdlIENvbnRyb2xsZXIn CiAgICBjbGFzcyAgICAgID0gbWFzcyBzdG9yYWdlCiAgICBzdWJjbGFzcyAg ID0gQVRBCmF0YXBjaTFAcGNpMDowOjMxOjI6CWNsYXNzPTB4MDEwMThmIGNh cmQ9MHgzNDcyODA4NiBjaGlwPTB4MjY4MDgwODYgcmV2PTB4MDkgaGRyPTB4 MDAKICAgIHZlbmRvciAgICAgPSAnSW50ZWwgQ29ycG9yYXRpb24nCiAgICBk ZXZpY2UgICAgID0gJzYzMXhFU0IvNjMyeEVTQi8zMTAwIFNlcmlhbCBBVEEg U3RvcmFnZSBDb250cm9sbGVyJwogICAgY2xhc3MgICAgICA9IG1hc3Mgc3Rv cmFnZQogICAgc3ViY2xhc3MgICA9IEFUQQogICAgY2FwIDAxWzcwXSA9IHBv d2Vyc3BlYyAyICBzdXBwb3J0cyBEMCBEMyAgY3VycmVudCBEMApub25lMUBw Y2kwOjA6MzE6MzoJY2xhc3M9MHgwYzA1MDAgY2FyZD0weDM0NzI4MDg2IGNo aXA9MHgyNjliODA4NiByZXY9MHgwOSBoZHI9MHgwMAogICAgdmVuZG9yICAg ICA9ICdJbnRlbCBDb3Jwb3JhdGlvbicKICAgIGRldmljZSAgICAgPSAnU01C dXMgQ29udHJvbGxlciAoNjMxeEVTQi82MzIxRVNCLzMxMDApJwogICAgY2xh c3MgICAgICA9IHNlcmlhbCBidXMKICAgIHN1YmNsYXNzICAgPSBTTUJ1cwpw Y2liMkBwY2kwOjE6MDowOgljbGFzcz0weDA2MDQwMCBjYXJkPTB4MzQ3Mjgw ODYgY2hpcD0weDM1MDA4MDg2IHJldj0weDAxIGhkcj0weDAxCiAgICB2ZW5k b3IgICAgID0gJ0ludGVsIENvcnBvcmF0aW9uJwogICAgZGV2aWNlICAgICA9 ICc2MzF4RVNCLzYzMnhFU0IgUENJZSBVcHN0cmVhbSBQb3J0JwogICAgY2xh c3MgICAgICA9IGJyaWRnZQogICAgc3ViY2xhc3MgICA9IFBDSS1QQ0kKICAg IGNhcCAxMFs0NF0gPSBQQ0ktRXhwcmVzcyAxIHVwc3RyZWFtIHBvcnQgbWF4 IGRhdGEgMTI4KDI1NikgbGluayB4OCh4OCkKICAgIGNhcCAwMVs3MF0gPSBw b3dlcnNwZWMgMiAgc3VwcG9ydHMgRDAgRDMgIGN1cnJlbnQgRDAKICAgIGNh cCAwZFs4MF0gPSBQQ0kgQnJpZGdlIGNhcmQ9MHgzNDcyODA4NgpwY2liNUBw Y2kwOjE6MDozOgljbGFzcz0weDA2MDQwMCBjYXJkPTB4MzQ3MjgwODYgY2hp cD0weDM1MGM4MDg2IHJldj0weDAxIGhkcj0weDAxCiAgICB2ZW5kb3IgICAg ID0gJ0ludGVsIENvcnBvcmF0aW9uJwogICAgZGV2aWNlICAgICA9ICc2MzF4 RVNCLzYzMnhFU0IgUENJZSB0byBQQ0ktWCBCcmlkZ2UnCiAgICBjbGFzcyAg ICAgID0gYnJpZGdlCiAgICBzdWJjbGFzcyAgID0gUENJLVBDSQogICAgY2Fw IDEwWzQ0XSA9IFBDSS1FeHByZXNzIDEgUENJIGJyaWRnZSBtYXggZGF0YSAx MjgoMjU2KSBsaW5rIHg4KHg4KQogICAgY2FwIDAxWzZjXSA9IHBvd2Vyc3Bl YyAyICBzdXBwb3J0cyBEMCBEMyAgY3VycmVudCBEMAogICAgY2FwIDBkWzgw XSA9IFBDSSBCcmlkZ2UgY2FyZD0weDM0NzI4MDg2CiAgICBjYXAgMDdbZDhd ID0gUENJLVggYnJpZGdlIApwY2liM0BwY2kwOjI6MDowOgljbGFzcz0weDA2 MDQwMCBjYXJkPTB4MzQ3MjgwODYgY2hpcD0weDM1MTA4MDg2IHJldj0weDAx IGhkcj0weDAxCiAgICB2ZW5kb3IgICAgID0gJ0ludGVsIENvcnBvcmF0aW9u JwogICAgZGV2aWNlICAgICA9ICc2MzF4RVNCLzYzMnhFU0IgUENJZSBEb3du c3RyZWFtIFBvcnQgRTEnCiAgICBjbGFzcyAgICAgID0gYnJpZGdlCiAgICBz dWJjbGFzcyAgID0gUENJLVBDSQogICAgY2FwIDEwWzQ0XSA9IFBDSS1FeHBy ZXNzIDEgZG93bnN0cmVhbSBwb3J0IG1heCBkYXRhIDEyOCgyNTYpIGxpbmsg eDQoeDgpCiAgICBjYXAgMDVbNjBdID0gTVNJIHN1cHBvcnRzIDEgbWVzc2Fn ZSwgNjQgYml0IAogICAgY2FwIDAxWzcwXSA9IHBvd2Vyc3BlYyAyICBzdXBw b3J0cyBEMCBEMyAgY3VycmVudCBEMAogICAgY2FwIDBkWzgwXSA9IFBDSSBC cmlkZ2UgY2FyZD0weDM0NzI4MDg2CnBjaWI0QHBjaTA6MjoyOjA6CWNsYXNz PTB4MDYwNDAwIGNhcmQ9MHgzNDcyODA4NiBjaGlwPTB4MzUxODgwODYgcmV2 PTB4MDEgaGRyPTB4MDEKICAgIHZlbmRvciAgICAgPSAnSW50ZWwgQ29ycG9y YXRpb24nCiAgICBkZXZpY2UgICAgID0gJzYzMXhFU0IvNjMyeEVTQiBQQ0ll IERvd25zdHJlYW0gUG9ydCBFMycKICAgIGNsYXNzICAgICAgPSBicmlkZ2UK ICAgIHN1YmNsYXNzICAgPSBQQ0ktUENJCiAgICBjYXAgMTBbNDRdID0gUENJ LUV4cHJlc3MgMSBkb3duc3RyZWFtIHBvcnQgbWF4IGRhdGEgMTI4KDI1Nikg bGluayB4NCh4NCkKICAgIGNhcCAwNVs2MF0gPSBNU0kgc3VwcG9ydHMgMSBt ZXNzYWdlLCA2NCBiaXQgCiAgICBjYXAgMDFbNzBdID0gcG93ZXJzcGVjIDIg IHN1cHBvcnRzIEQwIEQzICBjdXJyZW50IEQwCiAgICBjYXAgMGRbODBdID0g UENJIEJyaWRnZSBjYXJkPTB4MzQ3MjgwODYKZW0wQHBjaTA6MzowOjA6CWNs YXNzPTB4MDIwMDAwIGNhcmQ9MHgxMTVlODA4NiBjaGlwPTB4MTA1ZTgwODYg cmV2PTB4MDYgaGRyPTB4MDAKICAgIHZlbmRvciAgICAgPSAnSW50ZWwgQ29y cG9yYXRpb24nCiAgICBkZXZpY2UgICAgID0gJ0hQIE5DMzYwVCBQQ0llIERQ IEdpZ2FiaXQgU2VydmVyIEFkYXB0ZXIgKG4xZTUxMzIpJwogICAgY2xhc3Mg ICAgICA9IG5ldHdvcmsKICAgIHN1YmNsYXNzICAgPSBldGhlcm5ldAogICAg Y2FwIDAxW2M4XSA9IHBvd2Vyc3BlYyAyICBzdXBwb3J0cyBEMCBEMyAgY3Vy cmVudCBEMAogICAgY2FwIDA1W2QwXSA9IE1TSSBzdXBwb3J0cyAxIG1lc3Nh Z2UsIDY0IGJpdCBlbmFibGVkIHdpdGggMSBtZXNzYWdlCiAgICBjYXAgMTBb ZTBdID0gUENJLUV4cHJlc3MgMSBlbmRwb2ludCBtYXggZGF0YSAxMjgoMjU2 KSBsaW5rIHg0KHg0KQplbTFAcGNpMDozOjA6MToJY2xhc3M9MHgwMjAwMDAg Y2FyZD0weDExNWU4MDg2IGNoaXA9MHgxMDVlODA4NiByZXY9MHgwNiBoZHI9 MHgwMAogICAgdmVuZG9yICAgICA9ICdJbnRlbCBDb3Jwb3JhdGlvbicKICAg IGRldmljZSAgICAgPSAnSFAgTkMzNjBUIFBDSWUgRFAgR2lnYWJpdCBTZXJ2 ZXIgQWRhcHRlciAobjFlNTEzMiknCiAgICBjbGFzcyAgICAgID0gbmV0d29y awogICAgc3ViY2xhc3MgICA9IGV0aGVybmV0CiAgICBjYXAgMDFbYzhdID0g cG93ZXJzcGVjIDIgIHN1cHBvcnRzIEQwIEQzICBjdXJyZW50IEQwCiAgICBj YXAgMDVbZDBdID0gTVNJIHN1cHBvcnRzIDEgbWVzc2FnZSwgNjQgYml0IGVu YWJsZWQgd2l0aCAxIG1lc3NhZ2UKICAgIGNhcCAxMFtlMF0gPSBQQ0ktRXhw cmVzcyAxIGVuZHBvaW50IG1heCBkYXRhIDEyOCgyNTYpIGxpbmsgeDQoeDQp CmVtMkBwY2kwOjQ6MDowOgljbGFzcz0weDAyMDAwMCBjYXJkPTB4MzQ3Mjgw ODYgY2hpcD0weDEwOTY4MDg2IHJldj0weDAxIGhkcj0weDAwCiAgICB2ZW5k b3IgICAgID0gJ0ludGVsIENvcnBvcmF0aW9uJwogICAgZGV2aWNlICAgICA9 ICdJbnRlbCBQUk8vMTAwMCBFQiAoSW50ZWwgUFJPLzEwMDAgRUIpJwogICAg Y2xhc3MgICAgICA9IG5ldHdvcmsKICAgIHN1YmNsYXNzICAgPSBldGhlcm5l dAogICAgY2FwIDAxW2M4XSA9IHBvd2Vyc3BlYyAyICBzdXBwb3J0cyBEMCBE MyAgY3VycmVudCBEMAogICAgY2FwIDA1W2QwXSA9IE1TSSBzdXBwb3J0cyAx IG1lc3NhZ2UsIDY0IGJpdCBlbmFibGVkIHdpdGggMSBtZXNzYWdlCiAgICBj YXAgMTBbZTBdID0gUENJLUV4cHJlc3MgMSBlbmRwb2ludCBtYXggZGF0YSAx MjgoMjU2KSBsaW5rIHg0KHg0KQplbTNAcGNpMDo0OjA6MToJY2xhc3M9MHgw MjAwMDAgY2FyZD0weDM0NzI4MDg2IGNoaXA9MHgxMDk2ODA4NiByZXY9MHgw MSBoZHI9MHgwMAogICAgdmVuZG9yICAgICA9ICdJbnRlbCBDb3Jwb3JhdGlv bicKICAgIGRldmljZSAgICAgPSAnSW50ZWwgUFJPLzEwMDAgRUIgKEludGVs IFBSTy8xMDAwIEVCKScKICAgIGNsYXNzICAgICAgPSBuZXR3b3JrCiAgICBz dWJjbGFzcyAgID0gZXRoZXJuZXQKICAgIGNhcCAwMVtjOF0gPSBwb3dlcnNw ZWMgMiAgc3VwcG9ydHMgRDAgRDMgIGN1cnJlbnQgRDAKICAgIGNhcCAwNVtk MF0gPSBNU0kgc3VwcG9ydHMgMSBtZXNzYWdlLCA2NCBiaXQgZW5hYmxlZCB3 aXRoIDEgbWVzc2FnZQogICAgY2FwIDEwW2UwXSA9IFBDSS1FeHByZXNzIDEg ZW5kcG9pbnQgbWF4IGRhdGEgMTI4KDI1NikgbGluayB4NCh4NCkKdmdhcGNp MEBwY2kwOjc6MDowOgljbGFzcz0weDAzMDAwMCBjYXJkPTB4MjI3MTE3ODcg Y2hpcD0weDk1NGYxMDAyIHJldj0weDAwIGhkcj0weDAwCiAgICB2ZW5kb3Ig ICAgID0gJ0FUSSBUZWNobm9sb2dpZXMgSW5jLiAvIEFkdmFuY2VkIE1pY3Jv IERldmljZXMsIEluYy4nCiAgICBkZXZpY2UgICAgID0gJ0FUSSBSYWRlb24g SEQgNDM1MCAoUlY3MTApJwogICAgY2xhc3MgICAgICA9IGRpc3BsYXkKICAg IHN1YmNsYXNzICAgPSBWR0EKICAgIGNhcCAwMVs1MF0gPSBwb3dlcnNwZWMg MyAgc3VwcG9ydHMgRDAgRDEgRDIgRDMgIGN1cnJlbnQgRDAKICAgIGNhcCAx MFs1OF0gPSBQQ0ktRXhwcmVzcyAyIGxlZ2FjeSBlbmRwb2ludCBtYXggZGF0 YSAxMjgoMTI4KSBsaW5rIHgxNih4MTYpCiAgICBjYXAgMDVbYTBdID0gTVNJ IHN1cHBvcnRzIDEgbWVzc2FnZSwgNjQgYml0IApub25lMkBwY2kwOjc6MDox OgljbGFzcz0weDA0MDMwMCBjYXJkPTB4YWEzODE3ODcgY2hpcD0weGFhMzgx MDAyIHJldj0weDAwIGhkcj0weDAwCiAgICB2ZW5kb3IgICAgID0gJ0FUSSBU ZWNobm9sb2dpZXMgSW5jLiAvIEFkdmFuY2VkIE1pY3JvIERldmljZXMsIElu Yy4nCiAgICBjbGFzcyAgICAgID0gbXVsdGltZWRpYQogICAgc3ViY2xhc3Mg ICA9IEhEQQogICAgY2FwIDAxWzUwXSA9IHBvd2Vyc3BlYyAzICBzdXBwb3J0 cyBEMCBEMSBEMiBEMyAgY3VycmVudCBEMAogICAgY2FwIDEwWzU4XSA9IFBD SS1FeHByZXNzIDIgbGVnYWN5IGVuZHBvaW50IG1heCBkYXRhIDEyOCgxMjgp IGxpbmsgeDE2KHgxNikKICAgIGNhcCAwNVthMF0gPSBNU0kgc3VwcG9ydHMg MSBtZXNzYWdlLCA2NCBiaXQgCmVtNEBwY2kwOjExOjA6MDoJY2xhc3M9MHgw MjAwMDAgY2FyZD0weDExNWU4MDg2IGNoaXA9MHgxMDVlODA4NiByZXY9MHgw NiBoZHI9MHgwMAogICAgdmVuZG9yICAgICA9ICdJbnRlbCBDb3Jwb3JhdGlv bicKICAgIGRldmljZSAgICAgPSAnSFAgTkMzNjBUIFBDSWUgRFAgR2lnYWJp dCBTZXJ2ZXIgQWRhcHRlciAobjFlNTEzMiknCiAgICBjbGFzcyAgICAgID0g bmV0d29yawogICAgc3ViY2xhc3MgICA9IGV0aGVybmV0CiAgICBjYXAgMDFb YzhdID0gcG93ZXJzcGVjIDIgIHN1cHBvcnRzIEQwIEQzICBjdXJyZW50IEQw CiAgICBjYXAgMDVbZDBdID0gTVNJIHN1cHBvcnRzIDEgbWVzc2FnZSwgNjQg Yml0IGVuYWJsZWQgd2l0aCAxIG1lc3NhZ2UKICAgIGNhcCAxMFtlMF0gPSBQ Q0ktRXhwcmVzcyAxIGVuZHBvaW50IG1heCBkYXRhIDEyOCgyNTYpIGxpbmsg eDQoeDQpCmVtNUBwY2kwOjExOjA6MToJY2xhc3M9MHgwMjAwMDAgY2FyZD0w eDExNWU4MDg2IGNoaXA9MHgxMDVlODA4NiByZXY9MHgwNiBoZHI9MHgwMAog ICAgdmVuZG9yICAgICA9ICdJbnRlbCBDb3Jwb3JhdGlvbicKICAgIGRldmlj ZSAgICAgPSAnSFAgTkMzNjBUIFBDSWUgRFAgR2lnYWJpdCBTZXJ2ZXIgQWRh cHRlciAobjFlNTEzMiknCiAgICBjbGFzcyAgICAgID0gbmV0d29yawogICAg c3ViY2xhc3MgICA9IGV0aGVybmV0CiAgICBjYXAgMDFbYzhdID0gcG93ZXJz cGVjIDIgIHN1cHBvcnRzIEQwIEQzICBjdXJyZW50IEQwCiAgICBjYXAgMDVb ZDBdID0gTVNJIHN1cHBvcnRzIDEgbWVzc2FnZSwgNjQgYml0IGVuYWJsZWQg d2l0aCAxIG1lc3NhZ2UKICAgIGNhcCAxMFtlMF0gPSBQQ0ktRXhwcmVzcyAx IGVuZHBvaW50IG1heCBkYXRhIDEyOCgyNTYpIGxpbmsgeDQoeDQpCg== --0-303701128-1284710967=:42781-- From owner-freebsd-net@FreeBSD.ORG Fri Sep 17 12:50:03 2010 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E17E9106564A for ; Fri, 17 Sep 2010 12:50:03 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id CECEB8FC0A for ; Fri, 17 Sep 2010 12:50:03 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o8HCo3YL047125 for ; Fri, 17 Sep 2010 12:50:03 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o8HCo3h0047124; Fri, 17 Sep 2010 12:50:03 GMT (envelope-from gnats) Date: Fri, 17 Sep 2010 12:50:03 GMT Message-Id: <201009171250.o8HCo3h0047124@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: Pete French Cc: Subject: Re: kern/149804: [icmp] [panic] ICMP redirect on causes "panic: rtqkill route really not free" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Pete French List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 12:50:04 -0000 The following reply was made to PR kern/149804; it has been noted by GNATS. From: Pete French To: bug-followup@FreeBSD.org, petefrench@ticketswitch.com Cc: Subject: Re: kern/149804: [icmp] [panic] ICMP redirect on causes "panic: rtqkill route really not free" Date: Fri, 17 Sep 2010 13:44:14 +0100 Here is the patch I am currently using which makes the problem go away. = As states above, this was sent to me by Xin Li - it's not my own work! Index: sys/netinet/in_rmx.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/netinet/in_rmx.c (revision 211232) +++ sys/netinet/in_rmx.c (working copy) @@ -121,12 +121,13 @@ in_matroute(void *v_arg, struct radix_node_head *h struct radix_node *rn =3D rn_match(v_arg, head); struct rtentry *rt =3D (struct rtentry *)rn; =20 - /*XXX locking? */ - if (rt && rt->rt_refcnt =3D=3D 0) { /* this is first = reference */ + if (rt) { + RT_LOCK(rt); if (rt->rt_flags & RTPRF_OURS) { rt->rt_flags &=3D ~RTPRF_OURS; rt->rt_rmx.rmx_expire =3D 0; } + RT_UNLOCK(rt); } return rn; } From owner-freebsd-net@FreeBSD.ORG Fri Sep 17 14:07:20 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 784DA1065670 for ; Fri, 17 Sep 2010 14:07:20 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: from eu1sys200aog110.obsmtp.com (eu1sys200aog110.obsmtp.com [207.126.144.129]) by mx1.freebsd.org (Postfix) with SMTP id 0B1D08FC14 for ; Fri, 17 Sep 2010 14:07:18 +0000 (UTC) Received: from source ([63.174.175.251]) by eu1sys200aob110.postini.com ([207.126.147.11]) with SMTP ID DSNKTJN2FdFhoT9LsC+Ar7NwXyl/4Crz2vAP@postini.com; Fri, 17 Sep 2010 14:07:19 UTC Received: from [172.17.10.53] (unknown [172.17.10.53]) by bbbx3.usdmm.com (Postfix) with ESMTP id 7E555FD01A; Fri, 17 Sep 2010 14:07:17 +0000 (UTC) Message-ID: <4C93760B.8050206@tomjudge.com> Date: Fri, 17 Sep 2010 09:07:07 -0500 From: Tom Judge User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.12) Gecko/20100826 Lightning/1.0b1 Thunderbird/3.0.7 MIME-Version: 1.0 To: Vladimir Grigorov References: <4C923353.7090801@tomjudge.com> <1307024327.20100917121857@gmail.com> In-Reply-To: <1307024327.20100917121857@gmail.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: Fwd: Re: Strange FreeBSD behavior when trying to forward beetween ipsec crypted gif's. May be a problem with ICMP unreach packets at all X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 14:07:20 -0000 On 09/17/2010 03:18 AM, Vladimir Grigorov wrote: > greets all > > > >> If you take a look at icmp_error() in sys/netinet/ip_icmp.c you will see >> that icmp errors are not sent for packets that have been previously been >> decrypted by IPSec. >> > May be some misunderstandings happens. I have gif and ipsec. IPSEC mode is transport, that means, traffic encrypted only between gif's > outer addresses. As result, traffic in gif encrypted by encrypting ipip container. But I can view traffic on gif by tcpdump as on > regular interfaces. E.g. gif's inner traffic not processed by ipsec at all > > Consider you have a packet that looks something like this: |IP[1]|ESP|IP[2]|IP[3]|| 1) The packet enters ip_input() is validated against a policy and that its IP[1] header lists the router as the destination. 2) ip_input() passes the frame (mbuf) into ip_ipsec_input() which will return 0 and allow the frame to continue to be processed. 3) ip_input() then (eventually) calls esp_input() which in turn calls esp_input_cb() 4) esp_input_cb() does the decryption work and tags the mbuf containing the frame with M_DECRYPTED at this stage the frame in the mbuf will look like this: |IP[2]|IP[3]|| 5) esp_input_cp() passes the processed mbuf to ipsec_common_input_cb() which will redispatch the mbuf (frame) to in_gif_input() via the netisr queue. 6) in_gif_input() calls gif_input() to process the frame which will look like this: |IP[3]|| *Note: the mbuf this frame is stored in is the same mbuf as the original packet was received in by the NIC so still carries the flag M_DECRYPTED. 7) gif_input() re dispatches the mbuf via the netisr queue again. 8) Packet causes a call to icmp_error() in either ip_input() or ip_foarward() and ecmp_error() does not send the message as M_DECRYPTED is set. I have missed/glossed over a few steps here I feel, but in general I think from my 15 minutes reading the code this is how it works (or at least the important parts of it). Hope this helps. Tom -- TJU13-ARIN From owner-freebsd-net@FreeBSD.ORG Fri Sep 17 17:38:00 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BDE51106564A; Fri, 17 Sep 2010 17:38:00 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 24CA18FC0C; Fri, 17 Sep 2010 17:38:00 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id ABBEBA6A275; Sat, 18 Sep 2010 01:37:56 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id mEWRlFoyABMj; Sat, 18 Sep 2010 01:37:50 +0800 (CST) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id D0E1FA6A27F; Sat, 18 Sep 2010 01:37:48 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:subject:x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=uu/Hm3Dr4RZNXQRlI3JPItSuXfMOEdPPGs1pYGgefU0qZDbwkSoYhB+3uClmH9ULj wfFLZOlnMiAqABB3FJMxg== Message-ID: <4C93A762.40305@delphij.net> Date: Fri, 17 Sep 2010 10:37:38 -0700 From: Xin LI Organization: The Geek China Organization User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.12) Gecko/20100909 Thunderbird/3.0.7 ThunderBrowse/3.3.2 MIME-Version: 1.0 To: "freebsd-net@freebsd.org" , jfv@FreeBSD.org X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: em(4): discard frame w/o packet header (82547L on Supermicro X7SPA-H) on stable/8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 17:38:00 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, I have got a bit of em0: discard frame w/o packet header (and sometimes em1) and the system panics sometimes, the network traffic is not quite large. This is fairly fresh stable/8 running under amd64, with pf, altq, openvpn (tun mode) and ipv6 enabled. So far I haven't found a way to reliably trigger the panic though. Because of the watchdog (the system is now on production) setting the system doesn't seem to be able to complete a minidump, will try textdump and see if it works. Suggestion on debugging? Is this a known (fixed?) issue by the way? Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iQEcBAEBCAAGBQJMk6dhAAoJEATO+BI/yjfBDCUH/jJNsvmePB4Onq7rSgXvgJEu 556TFvobXiztiURW6imiis4OlfWS0wclRLuweCSxxkZpgpmUW7smNnXST/1i8njZ eGeSdQItTlJCV0Za6xVSzwLlOlWdbdQZ4uf1LxABbhBoRaSDXAmAp+7Zw4QyYiiN YdcEyCHcc8ko4a7H7dq+W01VVaMyyHUr8W6n+V7edXmMaQt2pMuZtMq4sIfe2Zlh rS56YqfIFBPCtQT7ywZfPhDmYwjatzhzjFfiwVOjCnn2eCSLw7ao0eRnHSv1IAgF atjD08QUi2XY1zHR4L+OB5n/5lnSfJdaiN1x/sbRx7a3VEBvF3MWyehA+dwgETo= =irb7 -----END PGP SIGNATURE----- From owner-freebsd-net@FreeBSD.ORG Fri Sep 17 17:42:10 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1CD2D1065673; Fri, 17 Sep 2010 17:42:10 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 7EAC58FC12; Fri, 17 Sep 2010 17:42:09 +0000 (UTC) Received: by wyb33 with SMTP id 33so3511683wyb.13 for ; Fri, 17 Sep 2010 10:42:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=jhq8E6Qy+pA0QsHQHFbggxOLGHjR/O4PSuvNZbOiY3k=; b=dPvZnNOdrbitsprFzMVTwAoL5I3QC+w+aWCV3TD8RTWxuDPFkZ/LvJTNG3Z9yUyevA rbsucP9I5F7H2gHvGyV2Cge+JmUkR9aqTpfB0bNyRz5nPeVSsKslFp7nMESQwMxlmbJE D/VsxDGh0v1jjm0BeS+XSD8A+lAoF0ytRPE7w= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=trdFIavL38JM1aBLCOhP6nbTokE/h882oC+Ibh3ZG+O7SFHqkNWpm3ub9V0dR+Qz1L BRzYRSZjcUh8so0kapQtQmo40wewmXXoaCsPKnDy3E4GjrjQsbvAVGkmrEypZAUXhbAG avOGIoLfKLbVmRdaZC8+ErVkeHRBLDhsiEakI= MIME-Version: 1.0 Received: by 10.216.10.5 with SMTP id 5mr4462143weu.81.1284745328326; Fri, 17 Sep 2010 10:42:08 -0700 (PDT) Received: by 10.216.48.20 with HTTP; Fri, 17 Sep 2010 10:42:08 -0700 (PDT) In-Reply-To: <4C93A762.40305@delphij.net> References: <4C93A762.40305@delphij.net> Date: Fri, 17 Sep 2010 10:42:08 -0700 Message-ID: From: Jack Vogel To: d@delphij.net Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: jfv@freebsd.org, "freebsd-net@freebsd.org" Subject: Re: em(4): discard frame w/o packet header (82547L on Supermicro X7SPA-H) on stable/8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 17:42:10 -0000 Put DDB/KDB into the kernel and get me the stack trace when this problem happens. Tell me exactly what the hardware is (pciconf). Jack On Fri, Sep 17, 2010 at 10:37 AM, Xin LI wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Hi, > > I have got a bit of em0: discard frame w/o packet header (and sometimes > em1) and the system panics sometimes, the network traffic is not quite > large. This is fairly fresh stable/8 running under amd64, with pf, > altq, openvpn (tun mode) and ipv6 enabled. > > So far I haven't found a way to reliably trigger the panic though. > Because of the watchdog (the system is now on production) setting the > system doesn't seem to be able to complete a minidump, will try textdump > and see if it works. > > Suggestion on debugging? Is this a known (fixed?) issue by the way? > > Cheers, > - -- > Xin LI http://www.delphij.net/ > FreeBSD - The Power to Serve! Live free or die > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.16 (FreeBSD) > > iQEcBAEBCAAGBQJMk6dhAAoJEATO+BI/yjfBDCUH/jJNsvmePB4Onq7rSgXvgJEu > 556TFvobXiztiURW6imiis4OlfWS0wclRLuweCSxxkZpgpmUW7smNnXST/1i8njZ > eGeSdQItTlJCV0Za6xVSzwLlOlWdbdQZ4uf1LxABbhBoRaSDXAmAp+7Zw4QyYiiN > YdcEyCHcc8ko4a7H7dq+W01VVaMyyHUr8W6n+V7edXmMaQt2pMuZtMq4sIfe2Zlh > rS56YqfIFBPCtQT7ywZfPhDmYwjatzhzjFfiwVOjCnn2eCSLw7ao0eRnHSv1IAgF > atjD08QUi2XY1zHR4L+OB5n/5lnSfJdaiN1x/sbRx7a3VEBvF3MWyehA+dwgETo= > =irb7 > -----END PGP SIGNATURE----- > From owner-freebsd-net@FreeBSD.ORG Fri Sep 17 17:54:21 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0C0F61065670; Fri, 17 Sep 2010 17:54:21 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id D0C308FC16; Fri, 17 Sep 2010 17:54:19 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 2C456A6A293; Sat, 18 Sep 2010 01:54:18 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id OCVsfSa9YcAT; Sat, 18 Sep 2010 01:53:55 +0800 (CST) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 375ADA6A275; Sat, 18 Sep 2010 01:53:50 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=UuENTaASEEPAk8T2Qpx4tvTervJR6UK4OaxuYO3WcP1rx6gSyziMW92SoiN4YZvF3 4ewi9atfqzSXqcY9O2f1w== Message-ID: <4C93AB2A.8010000@delphij.net> Date: Fri, 17 Sep 2010 10:53:46 -0700 From: Xin LI Organization: The Geek China Organization User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.12) Gecko/20100909 Thunderbird/3.0.7 ThunderBrowse/3.3.2 MIME-Version: 1.0 To: Jack Vogel References: <4C93A762.40305@delphij.net> In-Reply-To: X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: jfv@freebsd.org, "freebsd-net@freebsd.org" , d@delphij.net Subject: Re: em(4): discard frame w/o packet header (82547L on Supermicro X7SPA-H) on stable/8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 17:54:21 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2010/09/17 10:42, Jack Vogel wrote: > Put DDB/KDB into the kernel and get me the stack trace when this > problem happens. Tell me exactly what the hardware is (pciconf). Will do (I do have kdb/ddb in kernel but textdump still do minidump for some reason, I'm looking at the code to figure out why). pciconf -lv: Not sure if only em related is sufficient, the rest parts are below. ========================== em0@pci0:2:0:0: class=0x020000 card=0x060a15d9 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel 82574L Gigabit Ethernet Controller (82574L)' class = network subclass = ethernet em1@pci0:3:0:0: class=0x020000 card=0x060a15d9 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel 82574L Gigabit Ethernet Controller (82574L)' class = network subclass = ethernet ========================== hostb0@pci0:0:0:0: class=0x060000 card=0x060a15d9 chip=0xa0008086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' class = bridge subclass = HOST-PCI vgapci0@pci0:0:2:0: class=0x030000 card=0x060a15d9 chip=0xa0018086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' class = display subclass = VGA vgapci1@pci0:0:2:1: class=0x038000 card=0x060a15d9 chip=0xa0028086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' class = display uhci0@pci0:0:26:0: class=0x0c0300 card=0x060a15d9 chip=0x29378086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller' class = serial bus subclass = USB uhci1@pci0:0:26:1: class=0x0c0300 card=0x060a15d9 chip=0x29388086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller' class = serial bus subclass = USB uhci2@pci0:0:26:2: class=0x0c0300 card=0x060a15d9 chip=0x29398086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller' class = serial bus subclass = USB ehci0@pci0:0:26:7: class=0x0c0320 card=0x060a15d9 chip=0x293c8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB2 Enhanced Host Controller' class = serial bus subclass = USB pcib1@pci0:0:28:0: class=0x060400 card=0x060a15d9 chip=0x29408086 rev=0x02 hdr=0x01 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) PCIe Root Port 1' class = bridge subclass = PCI-PCI pcib2@pci0:0:28:4: class=0x060400 card=0x060a15d9 chip=0x29488086 rev=0x02 hdr=0x01 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) PCIe Root Port 5' class = bridge subclass = PCI-PCI pcib3@pci0:0:28:5: class=0x060400 card=0x060a15d9 chip=0x294a8086 rev=0x02 hdr=0x01 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) PCIe Root Port 6' class = bridge subclass = PCI-PCI uhci3@pci0:0:29:0: class=0x0c0300 card=0x060a15d9 chip=0x29348086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller' class = serial bus subclass = USB uhci4@pci0:0:29:1: class=0x0c0300 card=0x060a15d9 chip=0x29358086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller' class = serial bus subclass = USB uhci5@pci0:0:29:2: class=0x0c0300 card=0x060a15d9 chip=0x29368086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller' class = serial bus subclass = USB ehci1@pci0:0:29:7: class=0x0c0320 card=0x060a15d9 chip=0x293a8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) USB2 Enhanced Host Controller' class = serial bus subclass = USB pcib4@pci0:0:30:0: class=0x060401 card=0x060a15d9 chip=0x244e8086 rev=0x92 hdr=0x01 vendor = 'Intel Corporation' device = '82801 Family (ICH2/3/4/5/6/7/8/9,63xxESB) Hub Interface to PCI Bridge' class = bridge subclass = PCI-PCI isab0@pci0:0:31:0: class=0x060100 card=0x060a15d9 chip=0x29168086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IR (ICH9R) LPC Interface Controller' class = bridge subclass = PCI-ISA ahci0@pci0:0:31:2: class=0x010601 card=0x060a15d9 chip=0x29228086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801IB/IR/IH (ICH9 Family) 6 port SATA AHCI Controller' class = mass storage subclass = SATA ichsmb0@pci0:0:31:3: class=0x0c0500 card=0x060a15d9 chip=0x29308086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel(R) ICH9 Family SMBus Controller working fine with http://download.cnet.com/Chipset-Driver-Inte (8086)' class = serial bus subclass = SMBus Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iQEcBAEBCAAGBQJMk6sqAAoJEATO+BI/yjfB6k0H/jC3MjJZHuQLMA5Oa+wTiizz Rm5z/M6XC2pk4LP3LOWxDh3kfOYjg5paQnrKGURE5GfQOS2OjBYezyR0ZRg6V6ch KwbuQwE7jkZF5KzCsUBAtkBguXZT+9oS2+loS/7niSD8PzQRff0sIBZSXItXjSkx l5+njx+NfyAsE6saNRXy7oLTMXdECyp302AGbEZuW2q8xlIw3Ugl9VxeBc5Zq+PZ /7jEFEBRr0HUP1z6Xu7wyOIP5uwgF72pigRnIjgQhivB67vUzbn78ZrVgG1aG3yX tn63B1G/VNct2lA1aAhFpVQlP9yAtH02YDgx488zat+222iPMNzyZDDaWzv/jHA= =FAib -----END PGP SIGNATURE----- From owner-freebsd-net@FreeBSD.ORG Sat Sep 18 12:41:17 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4482F1065672; Sat, 18 Sep 2010 12:41:17 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from ns1temp.jnielsen.net (ns1temp.jnielsen.net [69.55.230.42]) by mx1.freebsd.org (Postfix) with ESMTP id 288138FC13; Sat, 18 Sep 2010 12:41:16 +0000 (UTC) Received: from [192.168.2.31] (jn@stealth.jnielsen.net [74.218.226.254]) (authenticated bits=0) by ns1temp.jnielsen.net (8.14.3/8.14.3) with ESMTP id o8ICfEpF079613 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Sat, 18 Sep 2010 08:41:15 -0400 (EDT) (envelope-from lists@jnielsen.net) Mime-Version: 1.0 (Apple Message framework v1081) Content-Type: text/plain; charset=us-ascii From: John Nielsen In-Reply-To: Date: Sat, 18 Sep 2010 08:41:13 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1F563B4F-B1DF-4829-966D-63264EC8AB61@jnielsen.net> <9EB9CCE4-2F4A-4653-BF11-066E3490F3AF@FreeBSD.org> To: Adrian Chadd X-Mailer: Apple Mail (2.1081) X-DCC--Metrics: ns1temp.jnielsen.net; whitelist Cc: freebsd-net@freebsd.org, Rui Paulo Subject: Re: Is 802.11n rate control being worked on? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Sep 2010 12:41:17 -0000 On Sep 14, 2010, at 11:16 PM, Adrian Chadd wrote: > I'm working on bringing over the changes from Linux ath9k into our > HAL. I'm slowly starting on bringing over simple bits and pieces but I > hope to eventually be able to bring over large chunks of the hardware > fiddling almost untouched. Since the current open Atheros development > by people with the docs is occuring in linux ath9k, being able to sync > against that is high up on my todo list. Sounds like a plan. The rate control is separate from the HAL, correct? = Would you be working on that as well? > Rui told me his main problem was lacking in reliable driver code to > actually make/receive 11n frames reliably. I'm hoping to just lift the > ath9k hardware code but that doesn't help with the needed net80211 > changes to support 11n. These would be driver-independent net80211 changes? > If you're happy to take over rui's 11n work, I'm happy working on > porting over ath9k driver/rate changes. I'm definitely happy to look at it. I'll wait until I know just how far = over my head this project is and how much time I'll have for it before I = commit to it though. :) What still needs to happen for 11n in net80211? = Are any of MIMO, 40MHz channels, 5GHz operation, etc candidates for = common code or is all of that handled in the driver? > On 8 September 2010 20:32, Rui Paulo wrote: >> On 7 Sep 2010, at 19:41, John Nielsen wrote: >>=20 >>> I am working on a network scenario which would benefit greatly from = the MIMO features and higher bandwidth of 802.11n. It's my understanding = that 11n is not fully supported in FreeBSD since there is no appropriate = rate control algorithm in the tree. Is that still the case? >>=20 >> I've worked on supporting 11n on ath_rate_sample but it's incomplete. >>=20 >>>=20 >>> I would _really_ like to run FreeBSD for this project, and I believe = the Atheros wireless cards I plan to use are supported by ath(4). I'd = like to find out what else needs to happen to complete the picture. I = may even go so far as to write some code myself. :) >>>=20 >>> Is anyone working on this at the moment? >>=20 >> Not really, I did some work in the past, but it's incomplete. >>=20 >>>=20 >>> Is it just the rate control that needs to be done or are there other = parts involved? Is MIMO separate? >>=20 >> We have MIMO on some non-Atheros drivers, but one of these drivers = (Ralink 11n) is not yet in the tree. >>=20 >>>=20 >>> Is there a detailed description of the missing pieces somewhere? Or = a not-very-detailed summary of where to look and what to read to get = started? >>=20 >> Not really. There's some interest from other FreeBSD committers to = get this going, so I'll let them chime in. >>=20 >> Regards, >> -- >> Rui Paulo >>=20 >>=20 >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Sat Sep 18 13:31:27 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 13E41106566B; Sat, 18 Sep 2010 13:31:27 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id D5B848FC0A; Sat, 18 Sep 2010 13:31:25 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA07210; Sat, 18 Sep 2010 16:31:24 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1OwxVk-000D5b-9Y; Sat, 18 Sep 2010 16:31:24 +0300 Message-ID: <4C94BF2B.8050008@freebsd.org> Date: Sat, 18 Sep 2010 16:31:23 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100912 Lightning/1.0b2 Thunderbird/3.1.3 MIME-Version: 1.0 To: freebsd-current@freebsd.org, freebsd-net@freebsd.org, freebsd-stable@freebsd.org X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: if_epair as module X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Sep 2010 13:31:27 -0000 Anybody uses if_epair compiled as _module_ on any platform other than amd64? If yes, could you please respond to me in private? Big thanks in advance. -- Andriy Gapon From owner-freebsd-net@FreeBSD.ORG Sat Sep 18 21:00:12 2010 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C5F041065672 for ; Sat, 18 Sep 2010 21:00:12 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 9384A8FC0A for ; Sat, 18 Sep 2010 21:00:12 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o8IL0CJC077786 for ; Sat, 18 Sep 2010 21:00:12 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o8IL0CKw077755; Sat, 18 Sep 2010 21:00:12 GMT (envelope-from gnats) Date: Sat, 18 Sep 2010 21:00:12 GMT Message-Id: <201009182100.o8IL0CKw077755@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: jhell Cc: Subject: Re: kern/146534: [icmp6] wrong source address in echo reply X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: jhell List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Sep 2010 21:00:12 -0000 The following reply was made to PR kern/146534; it has been noted by GNATS. From: jhell To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/146534: [icmp6] wrong source address in echo reply Date: Sat, 18 Sep 2010 16:52:21 -0400 This is a multi-part message in MIME format. --------------090403020508080503040902 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 This PR seems to have been left behind. While it has been, I have tested this patch for quite some time now with no downsides noted. While this in fact fixes the noted problem, attached is a patch that cleans this patch up a bit. With this patch applied is correct behavior. - -- jhell,v -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJMlSaFAAoJEJBXh4mJ2FR+Z4MH/0zUHH/fUIfmKlg96T60A6xW RN/RXql2Rz/FYHzkontth2vBbSfmchNQtpgytXKG7R2qklyInwVVcLKNsJaXVfWz y8tpETUZLxNuYk5zhvsSS5q4JRQO3rbz1UMg9+b2VubnJ0y9OHahd/wztDCgmSDn f6mQU2YJtuglgixp25BJMbE/Dqh5MA1QEh1vj1lFWD5323hW2GOFFC4Mo3hgoCFA QqBzW9GMFoyA749d24GZvvDYa11esuU8+uiNLb5oPuBEoULdIw913NLMrZSuB2Uv PeDCbCUGzIa9xnCZmk6sG9HfDiZaRDPlf3q6sUuVCNCCBYikuoD4SIr/qN4ToTs= =knfM -----END PGP SIGNATURE----- --------------090403020508080503040902 Content-Type: text/plain; name="sys_netinet6_icmp6.c.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="sys_netinet6_icmp6.c.patch" --- sys/netinet6/icmp6.c (revision 210616) +++ sys/netinet6/icmp6.c (working copy) @@ -2162,8 +2162,21 @@ } if ((srcp != NULL) && - (in6_addrscope(srcp) != in6_addrscope(&ip6->ip6_src))) - srcp = NULL; + (in6_addrscope(srcp) != in6_addrscope(&ip6->ip6_src))) { + struct sockaddr_in6 sin6; + + bzero(&sin6, sizeof(sin6)); + sin6.sin6_family = AF_INET6; + sin6.sin6_len = sizeof(sin6); + sin6.sin6_addr = origdst; + + ia = (struct in6_ifaddr *) + ifa_ifwithaddr((struct sockaddr *)&sin6); + + if (ia && (ia->ia6_flags & IN6_IFF_ANYCAST)) + srcp = NULL; + } + if (srcp == NULL) { int e; --------------090403020508080503040902 Content-Type: application/octet-stream; name="sys_netinet6_icmp6.c.patch.sig" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="sys_netinet6_icmp6.c.patch.sig" iQEcBAABAgAGBQJMlSaFAAoJEJBXh4mJ2FR+Qu8H/23Ge4zNAGRRdGKgcEtp4KD0eiLZ9ozq 8GAbnBDXnH6lV/mj1d/DrFOEmcs0ApXHtjlnvox47yHaNlEJ7ESA5V1wf44rsfxN2ZZYdTFE 5qVNsElCAZ0yAjPnBCTbeFDZKx0vq4DJz4JuHoDgb8rrQMaPUClljk/kRFB1BL6jPkau/Fsy f/aZlmEERrxGkMTmBtVbv8E5WTpNZ0X9hf7u+5UG0coQtFM6hnqHC07UNrm7+cqsKtlbbvBP k5JoUsUSjV1mUwfzy8GJD7wm9wFGotHAgEDbbax5fmNZI3gChBjK01nQWkC8k6dFR6lzO7sb hgoDbbEEnpgSBz5wS0p3Vl4= --------------090403020508080503040902-- From owner-freebsd-net@FreeBSD.ORG Sat Sep 18 23:00:11 2010 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A9951106566B for ; Sat, 18 Sep 2010 23:00:11 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 91D838FC14 for ; Sat, 18 Sep 2010 23:00:11 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o8IN0BUU000726 for ; Sat, 18 Sep 2010 23:00:11 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o8IN0BjR000708; Sat, 18 Sep 2010 23:00:11 GMT (envelope-from gnats) Date: Sat, 18 Sep 2010 23:00:11 GMT Message-Id: <201009182300.o8IN0BjR000708@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: jhell Cc: Subject: Re: kern/146534: [icmp6] wrong source address in echo reply X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: jhell List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Sep 2010 23:00:11 -0000 The following reply was made to PR kern/146534; it has been noted by GNATS. From: jhell To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/146534: [icmp6] wrong source address in echo reply Date: Sat, 18 Sep 2010 18:59:20 -0400 This is a multi-part message in MIME format. --------------090904000205030606040405 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 09/18/2010 16:52, jhell wrote: > > This PR seems to have been left behind. While it has been, I have tested > this patch for quite some time now with no downsides noted. > > While this in fact fixes the noted problem, attached is a patch that > cleans this patch up a bit. > > With this patch applied is correct behavior. > Last patch agrbled ;) New attached patch non garbled ;) -- jhell,v --------------090904000205030606040405 Content-Type: text/plain; name="sys_netinet6_icmp6.c.patch.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="sys_netinet6_icmp6.c.patch.txt" --- sys/netinet6/icmp6.c (revision 210616) +++ sys/netinet6/icmp6.c (working copy) @@ -2162,8 +2162,21 @@ } if ((srcp != NULL) && - (in6_addrscope(srcp) != in6_addrscope(&ip6->ip6_src))) - srcp = NULL; + (in6_addrscope(srcp) != in6_addrscope(&ip6->ip6_src))) { + struct sockaddr_in6 sin6; + + bzero(&sin6, sizeof(sin6)); + sin6.sin6_family = AF_INET6; + sin6.sin6_len = sizeof(sin6); + sin6.sin6_addr = origdst; + + ia = (struct in6_ifaddr *) + ifa_ifwithaddr((struct sockaddr *)&sin6); + + if (ia && (ia->ia6_flags & IN6_IFF_ANYCAST)) + srcp = NULL; + } + if (srcp == NULL) { int e; --------------090904000205030606040405--