From owner-freebsd-hardware@FreeBSD.ORG Sun Sep 2 02:44:02 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 04B24106566B for ; Sun, 2 Sep 2012 02:44:02 +0000 (UTC) (envelope-from ayoung@mosaicarchive.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id B02508FC15 for ; Sun, 2 Sep 2012 02:44:01 +0000 (UTC) Received: by obbun3 with SMTP id un3so9950774obb.13 for ; Sat, 01 Sep 2012 19:44:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:in-reply-to:references:date :message-id:subject:from:to:content-type:x-gm-message-state; bh=wjE9TgNPl0cua09dOP8YelWvJqEjIDpYUVVezRQIQKo=; b=f+epZJlLDnBZIGJq9DBrr4XSjaRkePTI0vzoFM0HVHgZC/rhu9x7vkWVAotTKI4REf /5w184+WDp6XNkkm2QudZhlclr55tHquWhYIgycjYw4JJs4ZpN/ZSxLHpBms/ZlDKBTv 5pJEPquYQcC9bG+4HGMqNRwZniPZnuzLsOVZPKbVTFkTvYXTELxUZGse+lLkSLf6+ZDb dGeCLtAbqck+uTdTFJp+gaSRsnLMEPTayrqkHwx6Y1FIjIEhE+iUWOlTE+AxMMLHUSAT 9FeLfhBkwxeeK/IPNj6tWtwpzyPXR3XUOaBbLHr6eFqxZJD1Wl0CYqjnakQ+uLcv0ZUB JUZw== MIME-Version: 1.0 Received: by 10.182.188.41 with SMTP id fx9mr11017030obc.92.1346553840644; Sat, 01 Sep 2012 19:44:00 -0700 (PDT) Received: by 10.76.174.38 with HTTP; Sat, 1 Sep 2012 19:44:00 -0700 (PDT) X-Originating-IP: [96.237.242.243] In-Reply-To: References: Date: Sat, 1 Sep 2012 22:44:00 -0400 Message-ID: From: Andy Young To: freebsd-hardware@freebsd.org X-Gm-Message-State: ALoCoQnwxhfd+ltDz+9aUj44INakUpDh7uM+IwW2pEqUVFr2mY9ryo3uHqwMEOylHTbvJePk8AzT Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Load testing knocks out network X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Sep 2012 02:44:02 -0000 I read through the driver man page, which is a great source of information. I see I'm using the Intel igb driver and it supports three tunables. Could I have exceeded the number of receive descriptors? What would the effect of this number being too low be? What about the Adaptive Interrupt Moderation? To clarify, I was simulating about 800 users simultaneously uploading files when the crash occurred. Thanks for any help or insights!! Andy NAME igb -- Intel(R) PRO/1000 PCI Express Gigabit Ethernet adapter driver LOADER TUNABLES Tunables can be set at the loader(8) prompt before booting the kernel or stored in loader.conf(5). hw.igb.rxd Number of receive descriptors allocated by the driver. The default value is 256. The minimum is 80, and the maximum is 4096. hw.igb.txd Number of transmit descriptors allocated by the driver. The default value is 256. The minimum is 80, and the maximum is 4096. hw.igb.enable_aim If set to 1, enable Adaptive Interrupt Moderation. The default is to enable Adaptive Interrupt Moderation. On Sat, Sep 1, 2012 at 4:14 PM, Andy Young wrote: > Last night one our servers went offline while I was load testing it. When > I got to the datacenter to check on it, the server seemed perfectly fine. > Everything was running on it, there were no panics or any other sign of a > hard crash. The only problem is the network was unreachable. I couldn't > connect to the box even from a laptop directly attached to the ethernet > port. I couldn't connect to anything from the box either. It was if the > network controller had seized up. I restarted netif and it didn't make a > difference. Rebooting the machine however, solved the issue and everything > went back to working great. I restarted the load testing and reproduced the > problem twice more this morning so at least its repeatable. It feels like a > network controller / driver issue to me for a couple reasons. First, the > problem affects the entire system. We're running FreeBSD 9 with about a > half dozen jails. Most of the jails are running Apache but the one I was > load testing was running Jetty. However, if it was my application code > crashing I would expect the problem to at least be isolated to the jail > that hosts it. Instead, the entire machine and all jails in it lose access > to the network. > > Apart from not being able to access the network, I don't see any other > signs of problems. This is the first major problem I've had to debug in > FreeBSD so I'm not a debugging expert by any means. There are no error > messages in /var/log/messages or dmesg apart from syslogd not being able to > reach the network. If anyone has ideas on where I can look for more > evidence of what is going wrong, I would really appreciate it. > > We're running FreeBSD 9.0-RELEASE-p3. The network controller is a Intel(R) > PRO/1000 Network Connection version - 2.2.5 configured with 6 ips using > aliases, five of which are used for jails. > > Thank you for the help!! > > Andy > > > -- Andrew Young Mosaic Storage Systems, Inc http://www.mosaicarchive.com/ Follow us on: Twitter , Facebook , Google Plus , Pinterest From owner-freebsd-hardware@FreeBSD.ORG Sun Sep 2 08:58:09 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3F20C106566B for ; Sun, 2 Sep 2012 08:58:09 +0000 (UTC) (envelope-from ragnar@gatorhole.com) Received: from maple.lonn.org (maple.lonn.org [109.228.153.253]) by mx1.freebsd.org (Postfix) with ESMTP id BF7358FC12 for ; Sun, 2 Sep 2012 08:58:08 +0000 (UTC) Received: from [10.0.1.16] (c213-100-153-38.cust.tele2.se [213.100.153.38]) (Authenticated sender: ragnar@gatorhole.com) by maple.lonn.org (Postfix) with ESMTPSA id D5C35735C6E for ; Sun, 2 Sep 2012 10:51:18 +0200 (CEST) Message-ID: <50431E04.5050207@gatorhole.com> Date: Sun, 02 Sep 2012 10:51:16 +0200 From: Ragnar Lonn User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120827 Thunderbird/15.0 MIME-Version: 1.0 To: freebsd-hardware@freebsd.org References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Load testing knocks out network X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Sep 2012 08:58:09 -0000 Hi Andy, I work for an online load testing service (loadimpact.com) and what we see is that the most common cause when a server crashes during a load test, is that it runs out of some vital system resource. Usually system memory, but network connections (sockets/file descriptors) is also a likely cause. You should have gotten some kind of error messages in the system log, but if the problem is easily repeatable I would set up monitoring of at least memory and file descriptors, and see if you are near the limits when the machine freezes. Regards, /Ragnar On 09/01/2012 10:14 PM, Andy Young wrote: > Last night one our servers went offline while I was load testing it. When I > got to the datacenter to check on it, the server seemed perfectly fine. > Everything was running on it, there were no panics or any other sign of a > hard crash. The only problem is the network was unreachable. I couldn't > connect to the box even from a laptop directly attached to the ethernet > port. I couldn't connect to anything from the box either. It was if the > network controller had seized up. I restarted netif and it didn't make a > difference. Rebooting the machine however, solved the issue and everything > went back to working great. I restarted the load testing and reproduced the > problem twice more this morning so at least its repeatable. It feels like a > network controller / driver issue to me for a couple reasons. First, the > problem affects the entire system. We're running FreeBSD 9 with about a > half dozen jails. Most of the jails are running Apache but the one I was > load testing was running Jetty. However, if it was my application code > crashing I would expect the problem to at least be isolated to the jail > that hosts it. Instead, the entire machine and all jails in it lose access > to the network. > > Apart from not being able to access the network, I don't see any other > signs of problems. This is the first major problem I've had to debug in > FreeBSD so I'm not a debugging expert by any means. There are no error > messages in /var/log/messages or dmesg apart from syslogd not being able to > reach the network. If anyone has ideas on where I can look for more > evidence of what is going wrong, I would really appreciate it. > > We're running FreeBSD 9.0-RELEASE-p3. The network controller is a Intel(R) > PRO/1000 Network Connection version - 2.2.5 configured with 6 ips using > aliases, five of which are used for jails. > > Thank you for the help!! > > Andy > _______________________________________________ > freebsd-hardware@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hardware > To unsubscribe, send any mail to "freebsd-hardware-unsubscribe@freebsd.org" From owner-freebsd-hardware@FreeBSD.ORG Mon Sep 3 02:41:16 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D6F6F1065700 for ; Mon, 3 Sep 2012 02:41:16 +0000 (UTC) (envelope-from ayoung@mosaicarchive.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 94AEC8FC0C for ; Mon, 3 Sep 2012 02:41:16 +0000 (UTC) Received: by obbun3 with SMTP id un3so11055267obb.13 for ; Sun, 02 Sep 2012 19:41:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:date:message-id:subject:from:to :content-type:x-gm-message-state; bh=No//9IpbPDUrdOz5JDEBi0TOml5w2PsW2rSU1kPyg6g=; b=kW5q7CTlvFzrluBCtwGPgnGMIhiGeHsyXkESQK0JqJzI5/ztuv1Fw9cLuPecaVai0F raUykG5pTllRp89/v7jyGoUyIC0rKzwlfjHglBW9vNGhoyllOVoZhXA+CSfpPTaOUgQw u1+42Q7OGgIKuiCDBHCSYmeJtbcf8eZOJdTThO58+5Cy6qmNqaT/u/0ksf//Xt7xmyT4 0yStscp/VYaC9h2lVFePMQ/UGbewJJq1DDRB2OjgHsXcokf5QYdEBnlzAEA0MWHUe7HW /J2q9jNurIyaRGXrA1qVoGMQzzG29lm9hxf9lLoudV3Y1YXKAhBP3TmQ5iaoeH1TyP8h SI+A== MIME-Version: 1.0 Received: by 10.60.22.103 with SMTP id c7mr12762314oef.75.1346640075634; Sun, 02 Sep 2012 19:41:15 -0700 (PDT) Received: by 10.76.174.38 with HTTP; Sun, 2 Sep 2012 19:41:15 -0700 (PDT) X-Originating-IP: [67.217.105.93] Date: Sun, 2 Sep 2012 22:41:15 -0400 Message-ID: From: Andy Young To: freebsd-hardware@freebsd.org X-Gm-Message-State: ALoCoQnckdfhn/Rh6QVvtoV7nHKS0qnlRGgiF0dsPmeQlDY3BD/nTaALasFOxVC0pEeJCC5mBEhY Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: What is rx_processing_limit sysctl for Intel igb NIC driver? X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 02:41:16 -0000 I am tuning our server that has an Intel 82576 gigabit NIC using the igb driver. I see a lot of posts on the net where people bump the rx_processing_limit sysctl from the default value of 100 to 4096. Can anyone tell me what this is intended to do? Thanks! Andy From owner-freebsd-hardware@FreeBSD.ORG Mon Sep 3 03:13:32 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AE299106566B for ; Mon, 3 Sep 2012 03:13:32 +0000 (UTC) (envelope-from jose.amengual@gmail.com) Received: from mail-wi0-f172.google.com (mail-wi0-f172.google.com [209.85.212.172]) by mx1.freebsd.org (Postfix) with ESMTP id 3C3B88FC0C for ; Mon, 3 Sep 2012 03:13:31 +0000 (UTC) Received: by wicr5 with SMTP id r5so2684274wic.13 for ; Sun, 02 Sep 2012 20:13:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=9oLJv2MnAq2XFRH3Ob4VRb6ATWHW4RHDnZjTnSDKaT4=; b=J7plbKT5Y8qRszvOfcRgJs9mevb5a+ajeLeRKF3udTC05WfS6yZKl6knP7sHHhli6f IUMYsxBfbCvP+qIKrx/Kc0EuqhJGYq9PQ/1GcrL3wm+H9V/4Ui55iseiao2Uqlpf9zKf g0MmHkHeBl+fWW9FJDl359s/YkAOlFZQelKckXe3MdLycno7tnfLjmxKji6s5XVnJyLW Cv2nyFGjwVoj8K3haWYWJ5GMz7a4eVRp4X50/zBpnrrVBYWsSgnsAP9crU18r1qUgskr OYgoTYbirU7i9OcaFvQaKdw1GX2Pi30XwQn7z4w6qa1nprUAT6pExFlZ8ZKnOT9UiFND eKeQ== MIME-Version: 1.0 Received: by 10.180.103.136 with SMTP id fw8mr20086287wib.20.1346642010949; Sun, 02 Sep 2012 20:13:30 -0700 (PDT) Received: by 10.227.92.24 with HTTP; Sun, 2 Sep 2012 20:13:30 -0700 (PDT) Received: by 10.227.92.24 with HTTP; Sun, 2 Sep 2012 20:13:30 -0700 (PDT) In-Reply-To: <50431E04.5050207@gatorhole.com> References: <50431E04.5050207@gatorhole.com> Date: Sun, 2 Sep 2012 20:13:30 -0700 Message-ID: From: "Pepe (Jose) Amengual" To: Ragnar Lonn Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hardware@freebsd.org Subject: Re: Load testing knocks out network X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 03:13:32 -0000 Maybe you should check vmstat -z while running the load testing to see if you get any errors. On Sep 2, 2012 1:58 AM, "Ragnar Lonn" wrote: > Hi Andy, > > I work for an online load testing service (loadimpact.com) and what we > see is that the most common cause when a server crashes during a load test, > is that it runs out of some vital system resource. Usually system memory, > but network connections (sockets/file descriptors) is also a likely cause. > > You should have gotten some kind of error messages in the system log, but > if the problem is easily repeatable I would set up monitoring of at least > memory and file descriptors, and see if you are near the limits when the > machine freezes. > > Regards, > > /Ragnar > > > On 09/01/2012 10:14 PM, Andy Young wrote: > >> Last night one our servers went offline while I was load testing it. When >> I >> got to the datacenter to check on it, the server seemed perfectly fine. >> Everything was running on it, there were no panics or any other sign of a >> hard crash. The only problem is the network was unreachable. I couldn't >> connect to the box even from a laptop directly attached to the ethernet >> port. I couldn't connect to anything from the box either. It was if the >> network controller had seized up. I restarted netif and it didn't make a >> difference. Rebooting the machine however, solved the issue and everything >> went back to working great. I restarted the load testing and reproduced >> the >> problem twice more this morning so at least its repeatable. It feels like >> a >> network controller / driver issue to me for a couple reasons. First, the >> problem affects the entire system. We're running FreeBSD 9 with about a >> half dozen jails. Most of the jails are running Apache but the one I was >> load testing was running Jetty. However, if it was my application code >> crashing I would expect the problem to at least be isolated to the jail >> that hosts it. Instead, the entire machine and all jails in it lose access >> to the network. >> >> Apart from not being able to access the network, I don't see any other >> signs of problems. This is the first major problem I've had to debug in >> FreeBSD so I'm not a debugging expert by any means. There are no error >> messages in /var/log/messages or dmesg apart from syslogd not being able >> to >> reach the network. If anyone has ideas on where I can look for more >> evidence of what is going wrong, I would really appreciate it. >> >> We're running FreeBSD 9.0-RELEASE-p3. The network controller is a Intel(R) >> PRO/1000 Network Connection version - 2.2.5 configured with 6 ips using >> aliases, five of which are used for jails. >> >> Thank you for the help!! >> >> Andy >> ______________________________**_________________ >> freebsd-hardware@freebsd.org mailing list >> http://lists.freebsd.org/**mailman/listinfo/freebsd-**hardware >> To unsubscribe, send any mail to "freebsd-hardware-unsubscribe@** >> freebsd.org " >> > > ______________________________**_________________ > freebsd-hardware@freebsd.org mailing list > http://lists.freebsd.org/**mailman/listinfo/freebsd-**hardware > To unsubscribe, send any mail to "freebsd-hardware-unsubscribe@** > freebsd.org " > From owner-freebsd-hardware@FreeBSD.ORG Mon Sep 3 04:04:03 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D8AF106566C for ; Mon, 3 Sep 2012 04:04:03 +0000 (UTC) (envelope-from ayoung@mosaicarchive.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id C80648FC17 for ; Mon, 3 Sep 2012 04:04:02 +0000 (UTC) Received: by obbun3 with SMTP id un3so11128809obb.13 for ; Sun, 02 Sep 2012 21:04:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:in-reply-to:references:date :message-id:subject:from:to:cc:content-type:x-gm-message-state; bh=9xVfCdLR9qbsSyhrLAFAfWEugegSq/LKfbyzr/nqTZA=; b=cpMdkMO8reQgVBkK87bduPPuyRPZ2D/c4culu8z01Vxu+ps2y0HSR7y53q5mZcLx7s i76WmYfxyKv8oR3oK4gMBygAzDuR+MBjwD9zDLnXvnOQPUPiBTkcRZ85+kM5FcWHll6Z l1bejnNar3RkYXenMsk0OQtj6g8XWhLzMZ2eG7nnwR4Tc5dLY14bDLNiCMoWkBJG7kQH tYgaen1a246yFLgbusgWDnJCP13/Rhg1rqUAA+Zw58mOzBLb2Zk1bhU8qRdUFnjVrQ5k 2YuxpBdXNYxptsWaH49YSHClW0yJL7l8mtJKGYG3zaCuV1OD0jsROrvX6+HbcRyvLbUH suhg== MIME-Version: 1.0 Received: by 10.182.111.74 with SMTP id ig10mr13457782obb.14.1346645042146; Sun, 02 Sep 2012 21:04:02 -0700 (PDT) Received: by 10.76.174.38 with HTTP; Sun, 2 Sep 2012 21:04:02 -0700 (PDT) X-Originating-IP: [96.237.242.243] In-Reply-To: References: <50431E04.5050207@gatorhole.com> Date: Mon, 3 Sep 2012 00:04:02 -0400 Message-ID: From: Andy Young To: "Pepe (Jose) Amengual" X-Gm-Message-State: ALoCoQnryYsA1/InB7crfa2UMo2ZoQIWXUhmy6LXmpQJKrSem5n04ZSKhPyXoRdsRZUB6HVz/csP Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hardware@freebsd.org Subject: Re: Load testing knocks out network X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 04:04:03 -0000 Hi Pepe, Thank you for the tip. I don't know how to interpret any of the output but I will dig into the documentation. Andy On Sun, Sep 2, 2012 at 11:13 PM, Pepe (Jose) Amengual < jose.amengual@gmail.com> wrote: > Maybe you should check vmstat -z while running the load testing to see if > you get any errors. > On Sep 2, 2012 1:58 AM, "Ragnar Lonn" wrote: > > > Hi Andy, > > > > I work for an online load testing service (loadimpact.com) and what we > > see is that the most common cause when a server crashes during a load > test, > > is that it runs out of some vital system resource. Usually system memory, > > but network connections (sockets/file descriptors) is also a likely > cause. > > > > You should have gotten some kind of error messages in the system log, but > > if the problem is easily repeatable I would set up monitoring of at least > > memory and file descriptors, and see if you are near the limits when the > > machine freezes. > > > > Regards, > > > > /Ragnar > > > > > > On 09/01/2012 10:14 PM, Andy Young wrote: > > > >> Last night one our servers went offline while I was load testing it. > When > >> I > >> got to the datacenter to check on it, the server seemed perfectly fine. > >> Everything was running on it, there were no panics or any other sign of > a > >> hard crash. The only problem is the network was unreachable. I couldn't > >> connect to the box even from a laptop directly attached to the ethernet > >> port. I couldn't connect to anything from the box either. It was if the > >> network controller had seized up. I restarted netif and it didn't make a > >> difference. Rebooting the machine however, solved the issue and > everything > >> went back to working great. I restarted the load testing and reproduced > >> the > >> problem twice more this morning so at least its repeatable. It feels > like > >> a > >> network controller / driver issue to me for a couple reasons. First, the > >> problem affects the entire system. We're running FreeBSD 9 with about a > >> half dozen jails. Most of the jails are running Apache but the one I was > >> load testing was running Jetty. However, if it was my application code > >> crashing I would expect the problem to at least be isolated to the jail > >> that hosts it. Instead, the entire machine and all jails in it lose > access > >> to the network. > >> > >> Apart from not being able to access the network, I don't see any other > >> signs of problems. This is the first major problem I've had to debug in > >> FreeBSD so I'm not a debugging expert by any means. There are no error > >> messages in /var/log/messages or dmesg apart from syslogd not being able > >> to > >> reach the network. If anyone has ideas on where I can look for more > >> evidence of what is going wrong, I would really appreciate it. > >> > >> We're running FreeBSD 9.0-RELEASE-p3. The network controller is a > Intel(R) > >> PRO/1000 Network Connection version - 2.2.5 configured with 6 ips using > >> aliases, five of which are used for jails. > >> > >> Thank you for the help!! > >> > >> Andy > >> ______________________________**_________________ > >> freebsd-hardware@freebsd.org mailing list > >> http://lists.freebsd.org/**mailman/listinfo/freebsd-**hardware< > http://lists.freebsd.org/mailman/listinfo/freebsd-hardware> > >> To unsubscribe, send any mail to "freebsd-hardware-unsubscribe@** > >> freebsd.org " > >> > > > > ______________________________**_________________ > > freebsd-hardware@freebsd.org mailing list > > http://lists.freebsd.org/**mailman/listinfo/freebsd-**hardware< > http://lists.freebsd.org/mailman/listinfo/freebsd-hardware> > > To unsubscribe, send any mail to "freebsd-hardware-unsubscribe@** > > freebsd.org " > > > _______________________________________________ > freebsd-hardware@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hardware > To unsubscribe, send any mail to "freebsd-hardware-unsubscribe@freebsd.org > " > -- Andrew Young Mosaic Storage Systems, Inc http://www.mosaicarchive.com/ Follow us on: Twitter , Facebook , Google Plus , Pinterest From owner-freebsd-hardware@FreeBSD.ORG Mon Sep 3 04:14:14 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 70082106566B for ; Mon, 3 Sep 2012 04:14:14 +0000 (UTC) (envelope-from ayoung@mosaicarchive.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 2635D8FC0A for ; Mon, 3 Sep 2012 04:14:13 +0000 (UTC) Received: by obbun3 with SMTP id un3so11138492obb.13 for ; Sun, 02 Sep 2012 21:14:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:in-reply-to:references:date :message-id:subject:from:to:cc:content-type:x-gm-message-state; bh=CljdvpjTIQrYxzMdVn71KGg/wzVmTewsQBIxYe7YiUQ=; b=Y3hBfFA9tQnM7Sf/r2AJWwL8hddBrSb+No2nbCd9DQZr68Jh3SXHQj+MuEqE3jbfCw nGwxA1dPbhCR+Uu2qmH6VlZhGjI4S3hmZQ71gfgcqxV2FlH1J2ypVu0s8lI7v0PcqoKV 6oo+fTavkg2NW9vlQB4qIP0fQg7tZPCwDIVMQsnkzuT0yTVRl5sM3kilfIXJp39wbtmQ xEoq1CUN61v/RumB/zq0UuObz1Fmg5LneyrTctGWB5Ay2J3Pl1+K+gMvVzcAWINijCMm p2/QxDIjQFjBZpXnrJzmpcFML1tlI20yjp7NBNY5OmMb2ih6IVHBLoj54+IBqmImS33M baSg== MIME-Version: 1.0 Received: by 10.182.73.65 with SMTP id j1mr13275615obv.42.1346645653410; Sun, 02 Sep 2012 21:14:13 -0700 (PDT) Received: by 10.76.174.38 with HTTP; Sun, 2 Sep 2012 21:14:13 -0700 (PDT) X-Originating-IP: [96.237.242.243] In-Reply-To: References: Date: Mon, 3 Sep 2012 00:14:13 -0400 Message-ID: From: Andy Young To: Ragnar Lonn X-Gm-Message-State: ALoCoQngxsnLS2msGmQ67OjvqOnU0+SrNl3MZejKbRQJpbVZljpVH915rbYcO7D9we53EYM7e3cZ Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hardware@freebsd.org Subject: Re: Load testing knocks out network X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 04:14:14 -0000 Hi Ragnar, Thank you for the reply. That makes a lot of sense. I think the resources at risk had to do with the low level details of the network card. I experimented tonight with bumping the hw.igb.rxd and hw.igb.txd tunable parameters of the NIC driver to their max value of 4096 (the default was 256). This seems to have resolved the issue. Before bumping their values, my load test was crashing the network at about 350 simultaneous connections. The behavior I witnessed was the application server (jetty) would seize up first and I could see it was no longer responding through my ssh connections. If I killed it off right away, all of the connections got closed and everything went back to being fine. If I left it in that state for 30 seconds or so, the system became unrecoverable. The connections remained open according to netstat even after I had closed the server and client processes. Short of rebooting, nothing I did would close the connections down. After bumping the rxd and txd parameters as well as kern.ipc.nmbclusters, the problem seems to have gone away. I can now successfully simulate over 800 simultaneous connections and it hasn't crashed since. To be honest, I don't know what the rxd and txd parameters do but it seems to have helped. Andy On Sep 2, 2012 1:58 AM, "Ragnar Lonn" wrote: > Hi Andy, > > I work for an online load testing service (loadimpact.com) and what we > see is that the most common cause when a server crashes during a load test, > is that it runs out of some vital system resource. Usually system memory, > but network connections (sockets/file descriptors) is also a likely cause. > > You should have gotten some kind of error messages in the system log, but > if the problem is easily repeatable I would set up monitoring of at least > memory and file descriptors, and see if you are near the limits when the > machine freezes. > > Regards, > > /Ragnar On Sat, Sep 1, 2012 at 10:44 PM, Andy Young wrote: > I read through the driver man page, which is a great source of > information. I see I'm using the Intel igb driver and it supports three > tunables. Could I have exceeded the number of receive descriptors? What > would the effect of this number being too low be? What about the Adaptive > Interrupt Moderation? > > To clarify, I was simulating about 800 users simultaneously uploading > files when the crash occurred. > > Thanks for any help or insights!! > > Andy > > NAME > igb -- Intel(R) PRO/1000 PCI Express Gigabit Ethernet adapter driver > > LOADER TUNABLES > Tunables can be set at the loader(8) prompt before booting the kernel > or > stored in loader.conf(5). > > hw.igb.rxd > Number of receive descriptors allocated by the driver. The > default value is 256. The minimum is 80, and the maximum is > 4096. > > hw.igb.txd > Number of transmit descriptors allocated by the driver. The > default value is 256. The minimum is 80, and the maximum is > 4096. > > hw.igb.enable_aim > If set to 1, enable Adaptive Interrupt Moderation. The > default > is to enable Adaptive Interrupt Moderation. > > > On Sat, Sep 1, 2012 at 4:14 PM, Andy Young wrote: > >> Last night one our servers went offline while I was load testing it. When >> I got to the datacenter to check on it, the server seemed perfectly fine. >> Everything was running on it, there were no panics or any other sign of a >> hard crash. The only problem is the network was unreachable. I couldn't >> connect to the box even from a laptop directly attached to the ethernet >> port. I couldn't connect to anything from the box either. It was if the >> network controller had seized up. I restarted netif and it didn't make a >> difference. Rebooting the machine however, solved the issue and everything >> went back to working great. I restarted the load testing and reproduced the >> problem twice more this morning so at least its repeatable. It feels like a >> network controller / driver issue to me for a couple reasons. First, the >> problem affects the entire system. We're running FreeBSD 9 with about a >> half dozen jails. Most of the jails are running Apache but the one I was >> load testing was running Jetty. However, if it was my application code >> crashing I would expect the problem to at least be isolated to the jail >> that hosts it. Instead, the entire machine and all jails in it lose access >> to the network. >> >> Apart from not being able to access the network, I don't see any other >> signs of problems. This is the first major problem I've had to debug in >> FreeBSD so I'm not a debugging expert by any means. There are no error >> messages in /var/log/messages or dmesg apart from syslogd not being able to >> reach the network. If anyone has ideas on where I can look for more >> evidence of what is going wrong, I would really appreciate it. >> >> We're running FreeBSD 9.0-RELEASE-p3. The network controller is a >> Intel(R) PRO/1000 Network Connection version - 2.2.5 configured with 6 ips >> using aliases, five of which are used for jails. >> >> Thank you for the help!! >> >> Andy >> >> >> > > > -- > Andrew Young > Mosaic Storage Systems, Inc > http://www.mosaicarchive.com/ > > Follow us on: > Twitter , Facebook > , Google Plus > , Pinterest > > -- Andrew Young Mosaic Storage Systems, Inc http://www.mosaicarchive.com/ Follow us on: Twitter , Facebook , Google Plus , Pinterest From owner-freebsd-hardware@FreeBSD.ORG Mon Sep 3 08:12:53 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D0476106576F for ; Mon, 3 Sep 2012 08:12:53 +0000 (UTC) (envelope-from ragnar@gatorhole.com) Received: from maple.lonn.org (maple.lonn.org [109.228.153.253]) by mx1.freebsd.org (Postfix) with ESMTP id 0FE558FC12 for ; Mon, 3 Sep 2012 08:12:52 +0000 (UTC) Received: from [192.168.32.123] (unknown [85.24.141.131]) (Authenticated sender: ragnar@gatorhole.com) by maple.lonn.org (Postfix) with ESMTPSA id 681F7735C5B; Mon, 3 Sep 2012 10:12:51 +0200 (CEST) Message-ID: <50446681.2080307@gatorhole.com> Date: Mon, 03 Sep 2012 10:12:49 +0200 From: Ragnar Lonn User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120827 Thunderbird/15.0 MIME-Version: 1.0 To: Andy Young References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hardware@freebsd.org Subject: Re: Load testing knocks out network X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 08:12:53 -0000 Hi Andy, It sounds as if your problem is more related to the NIC driver then, I guess. I just realized something else that I forgot to mention regarding crash/freeze causes: network buffers. It's an out-of-memory problem, but a bit more specific. When you have a ton of open TCP connections to a host, that host will allocate a lot of (kernel) memory for TCP transmit/receive buffers. In newer *Linux* kernels, this memory is being allocated in an adaptive manner - i.e. the kernel only allocates a small amount of memory to each TCP buffer, and then increases it as necessary (per connection, depending on transfer speed and network delay to the other peer). Older kernels, however, will allocate a fixed amount per socket, which can quickly eat up all available kernel memory. I think I actually discussed this with FreeBSD developers a while ago (on this list even?), and they told me the FreeBSD kernel can only allocate max 2GB of kernel memory. I don't know if it allocates network buffers dynamically (i.e. as much memory as is necessary for each socket/connection) but 2GB is not a lot if each connection uses up e.g. 100K buffer memory. If you have e.g. 1GB available to network buffers, it means a max limit of 10k simultaneous connections on a server, regardless of how much memory it has. Regards, /Ragnar On 09/03/2012 06:14 AM, Andy Young wrote: > Hi Ragnar, > > Thank you for the reply. That makes a lot of sense. I think the > resources at risk had to do with the low level details of the network > card. I experimented tonight with bumping the hw.igb.rxd and > hw.igb.txd tunable parameters of the NIC driver to their max value of > 4096 (the default was 256). This seems to have resolved the issue. > Before bumping their values, my load test was crashing the network at > about 350 simultaneous connections. The behavior I witnessed was the > application server (jetty) would seize up first and I could see it was > no longer responding through my ssh connections. If I killed it off > right away, all of the connections got closed and everything went back > to being fine. If I left it in that state for 30 seconds or so, the > system became unrecoverable. The connections remained open according > to netstat even after I had closed the server and client processes. > Short of rebooting, nothing I did would close the connections down. > After bumping the rxd and txd parameters as well as > kern.ipc.nmbclusters, the problem seems to have gone away. I can now > successfully simulate over 800 simultaneous connections and it hasn't > crashed since. To be honest, I don't know what the rxd and txd > parameters do but it seems to have helped. > > Andy > > On Sep 2, 2012 1:58 AM, "Ragnar Lonn" > wrote: > > > Hi Andy, > > > > I work for an online load testing service (loadimpact.com > ) and what we > > see is that the most common cause when a server crashes during a load > test, > > is that it runs out of some vital system resource. Usually system memory, > > but network connections (sockets/file descriptors) is also a likely > cause. > > > > You should have gotten some kind of error messages in the system log, but > > if the problem is easily repeatable I would set up monitoring of at least > > memory and file descriptors, and see if you are near the limits when the > > machine freezes. > > > > Regards, > > > > /Ragnar > > On Sat, Sep 1, 2012 at 10:44 PM, Andy Young > wrote: > > I read through the driver man page, which is a great source of > information. I see I'm using the Intel igb driver and it supports > three tunables. Could I have exceeded the number of receive > descriptors? What would the effect of this number being too low > be? What about the Adaptive Interrupt Moderation? > > To clarify, I was simulating about 800 users simultaneously > uploading files when the crash occurred. > > Thanks for any help or insights!! > > Andy > > NAME > igb -- Intel(R) PRO/1000 PCI Express Gigabit Ethernet adapter > driver > > LOADER TUNABLES > Tunables can be set at the loader(8) prompt before booting > the kernel or > stored in loader.conf(5). > > hw.igb.rxd > Number of receive descriptors allocated by the > driver. The > default value is 256. The minimum is 80, and the > maximum is > 4096. > > hw.igb.txd > Number of transmit descriptors allocated by the > driver. The > default value is 256. The minimum is 80, and the > maximum is > 4096. > > hw.igb.enable_aim > If set to 1, enable Adaptive Interrupt Moderation. > The default > is to enable Adaptive Interrupt Moderation. > > > On Sat, Sep 1, 2012 at 4:14 PM, Andy Young > > wrote: > > Last night one our servers went offline while I was load > testing it. When I got to the datacenter to check on it, the > server seemed perfectly fine. Everything was running on it, > there were no panics or any other sign of a hard crash. The > only problem is the network was unreachable. I couldn't > connect to the box even from a laptop directly attached to the > ethernet port. I couldn't connect to anything from the box > either. It was if the network controller had seized up. I > restarted netif and it didn't make a difference. Rebooting the > machine however, solved the issue and everything went back to > working great. I restarted the load testing and reproduced the > problem twice more this morning so at least its repeatable. It > feels like a network controller / driver issue to me for a > couple reasons. First, the problem affects the entire system. > We're running FreeBSD 9 with about a half dozen jails. Most of > the jails are running Apache but the one I was load testing > was running Jetty. However, if it was my application code > crashing I would expect the problem to at least be isolated to > the jail that hosts it. Instead, the entire machine and all > jails in it lose access to the network. > > Apart from not being able to access the network, I don't see > any other signs of problems. This is the first major problem > I've had to debug in FreeBSD so I'm not a debugging expert by > any means. There are no error messages in /var/log/messages or > dmesg apart from syslogd not being able to reach the > network. If anyone has ideas on where I can look for more > evidence of what is going wrong, I would really appreciate it. > > We're running FreeBSD 9.0-RELEASE-p3. The network controller > is a Intel(R) PRO/1000 Network Connection version - 2.2.5 > configured with 6 ips using aliases, five of which are used > for jails. > > Thank you for the help!! > > Andy > > > > > > -- > Andrew Young > Mosaic Storage Systems, Inc > http://www.mosaicarchive.com/ > > Follow us on: > Twitter , Facebook > , Google Plus > , > Pinterest > > > > > -- > Andrew Young > Mosaic Storage Systems, Inc > http://www.mosaicarchive.com/ > > Follow us on: > Twitter , Facebook > , Google Plus > , > Pinterest > From owner-freebsd-hardware@FreeBSD.ORG Mon Sep 3 11:09:29 2012 Return-Path: Delivered-To: freebsd-hardware@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA46A106564A for ; Mon, 3 Sep 2012 11:09:29 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 9A2258FC1C for ; Mon, 3 Sep 2012 11:09:29 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q83B9Tlm045394 for ; Mon, 3 Sep 2012 11:09:29 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q83B9Rk0045035 for freebsd-hardware@FreeBSD.org; Mon, 3 Sep 2012 11:09:27 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 3 Sep 2012 11:09:27 GMT Message-Id: <201209031109.q83B9Rk0045035@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-hardware@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-hardware@FreeBSD.org X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 11:09:29 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/156241 hardware [mfi] 'zfs send' does not prevents disks to suspend if 1 problem total. From owner-freebsd-hardware@FreeBSD.ORG Mon Sep 3 11:36:21 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4CC2A1065678 for ; Mon, 3 Sep 2012 11:36:21 +0000 (UTC) (envelope-from dantavious313@gmail.com) Received: from mail-gg0-f182.google.com (mail-gg0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 01B368FC14 for ; Mon, 3 Sep 2012 11:36:20 +0000 (UTC) Received: by ggnk4 with SMTP id k4so923799ggn.13 for ; Mon, 03 Sep 2012 04:36:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:subject:date:user-agent:mime-version:content-type :content-transfer-encoding:message-id; bh=5pbUz/qiGeELRq79TW+28bCs+trFWJn/3sHIPHf0zwI=; b=rGpiQUT9hKMPh9wgHYag/HO8naCQquYhUofBRpYO1ImIdkYXa4AfplQNZuCg8AForZ AjFj2fXV23LeTQ792HPjXbKP4f8NVHp0rYkgJDNgJNhPeVWgyos4KKHjrRI+RhPwGWyX 5B4WKXCYgWRkl1J20USNK96yoNtmR5gr2a2Smz5J1ObUZyHyyF4svY3s+ICuxMMv3Nam SU8ee5UmSMcPn38mAGM6DtrvrREPfaWd4eBMEve6LDXo/KTsYhpyTKsoYuqaXJharixW ukMg48rk7Idu6yk7ol0bjAo4zRAp8gX3PFB4Rumh2koansNGe+r7XXy76dZiP32r6h6z hr4w== Received: by 10.236.153.9 with SMTP id e9mr14816838yhk.32.1346672180335; Mon, 03 Sep 2012 04:36:20 -0700 (PDT) Received: from joker.localnet (c-71-226-137-213.hsd1.ga.comcast.net. [71.226.137.213]) by mx.google.com with ESMTPS id w1sm11714873anm.8.2012.09.03.04.36.19 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 03 Sep 2012 04:36:19 -0700 (PDT) From: Derrick Edwards To: freebsd-hardware@freebsd.org Date: Mon, 3 Sep 2012 07:36:13 -0400 User-Agent: KMail/1.13.7 (FreeBSD/10.0-CURRENT; KDE/4.8.4; amd64; ; ) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart8163904.NqEuOU1J9F"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <201209030736.18334.dantavious313@gmail.com> Subject: Atheros AR8162/8166/8168 PCI-E Fast Ethernet Support X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 11:36:21 -0000 --nextPart8163904.NqEuOU1J9F Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi,=20 I purchased a Toshiba Satellite P875-S7200. I can not use the onboard ether= net=20 port because there is not a driver associated with it. I was wondering if a= n=20 driver was forthcoming supporting the subject chipsets. Also, are there any= =20 work arounds. =46reeBSD 10.0-CURRENT #1: Sun Sep 2 06:02:47 EDT 2012 none2@pci0:1:0:0: class=3D0x020000 card=3D0xff1e1179 chip=3D0x1090196= 9=20 rev=3D0x10 hdr=3D0x00 vendor =3D 'Atheros Communications' class =3D network subclass =3D ethernet bar [10] =3D type Memory, range 64, base 0xb8600000, size 262144, ena= bled bar [18] =3D type I/O Port, range 32, base 0x2000, size 128, enabled cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 10[58] =3D PCI-Express 1 endpoint max data 128(4096) link x1(x1) cap 05[c0] =3D MSI supports 16 messages, 64 bit, vector masks=20 cap 11[d8] =3D MSI-X supports 16 messages in map 0x10 ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 0 corrected ecap 0003[180] =3D Serial 1 ff262ff500266cff V/r Derrick --nextPart8163904.NqEuOU1J9F Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQEcBAABAgAGBQJQRJYyAAoJEPCFnSUiJPAUT+oIAI4GcleAFDBdXEl9OywN++jN nuKUFwa66tIuyxl1lVzN/TzM28Vdw3lrZXlG8wJxuLDbjDKouMr6fS6hTxbKUj/7 0fU5IQckuL5DshnnOdDoo5KOYW1EecwvfG4qbJvIAJTSupbpiSFn9kTZxfzWoxru /uUXD3QZtsJvxRgOmw1KHNbGsOUOpry51nLbgBjXWaxLnghIs2zOekVNbkzmyeou 076rHh+C6VBIYecN/tGRVpJ0NlKppCXCsDVqfAfYysXImIK9HOyeRohKfA39VU4v mLq7UP2SP511k6pQ7ds8Fj+UU5fAwhUW81zwdyGETAsaMq5jgmcEBt5TqmoYp8s= =+Cyv -----END PGP SIGNATURE----- --nextPart8163904.NqEuOU1J9F-- From owner-freebsd-hardware@FreeBSD.ORG Mon Sep 3 21:05:24 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 49FE4106566B for ; Mon, 3 Sep 2012 21:05:24 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (host-122-100-2-194.octopus.com.au [122.100.2.194]) by mx1.freebsd.org (Postfix) with ESMTP id B3CE48FC17 for ; Mon, 3 Sep 2012 21:05:23 +0000 (UTC) Received: from aspire.rulingia.com (12.58.233.220.static.exetel.com.au [220.233.58.12]) by vps.rulingia.com (8.14.5/8.14.5) with ESMTP id q83L51e2035046 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 4 Sep 2012 07:05:15 +1000 (EST) (envelope-from peter@rulingia.com) Received: from aspire.rulingia.com (localhost [127.0.0.1]) by aspire.rulingia.com (8.14.5/8.14.5) with ESMTP id q83L4sr7005161 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 4 Sep 2012 07:04:54 +1000 (EST) (envelope-from peter@aspire.rulingia.com) Received: (from peter@localhost) by aspire.rulingia.com (8.14.5/8.14.5/Submit) id q83L4pv5005160; Tue, 4 Sep 2012 07:04:51 +1000 (EST) (envelope-from peter) Date: Tue, 4 Sep 2012 07:04:50 +1000 From: Peter Jeremy To: Ragnar Lonn Message-ID: <20120903210450.GC2654@aspire.rulingia.com> References: <50446681.2080307@gatorhole.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Qbvjkv9qwOGw/5Fx" Content-Disposition: inline In-Reply-To: <50446681.2080307@gatorhole.com> X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hardware@freebsd.org Subject: Re: Load testing knocks out network X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 21:05:24 -0000 --Qbvjkv9qwOGw/5Fx Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2012-Sep-03 10:12:49 +0200, Ragnar Lonn wrote: >transmit/receive buffers. In newer *Linux* kernels, this memory is being= =20 >allocated in an adaptive manner - i.e. the kernel only allocates a small= =20 >amount of memory to each TCP buffer, and then increases it as necessary=20 >(per connection, depending on transfer speed and network delay to the=20 >other peer). FreeBSD does this as well, though I don't recall when this was added. >I think I actually discussed this with FreeBSD developers a while ago=20 >(on this list even?), and they told me the FreeBSD kernel can only=20 >allocate max 2GB of kernel memory. This is only true on 32-bit kernels. FreeBSD uses a single address space so both kernel and userland need to fit into 4GB on 32-bit systems. On 64-bit systems, KVM is less constrained (it's ~550GB on my amd64). You can check sysctl's vm.kvm_free and vm.kvm_size for exact figures. >100K buffer memory. If you have e.g. 1GB available to network buffers,=20 >it means a max limit of 10k simultaneous connections on a server,=20 >regardless of how much memory it has. If you want a system to usefully cope with 10K network connections, you will probably want to be running amd64 anyway. That said, Rod Grimes was achieving between 100K and 1M TCP connections to FreeBSD i386 systems in the 1990's. --=20 Peter Jeremy --Qbvjkv9qwOGw/5Fx Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlBFG3IACgkQ/opHv/APuIde8gCgtEiA5BjFU3khGRk6Ha12suN8 tf8AoLbVrQszZHQjc5ofjS/ywURg8X4J =AwPO -----END PGP SIGNATURE----- --Qbvjkv9qwOGw/5Fx-- From owner-freebsd-hardware@FreeBSD.ORG Mon Sep 3 21:57:11 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 440C6106564A for ; Mon, 3 Sep 2012 21:57:11 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (host-122-100-2-194.octopus.com.au [122.100.2.194]) by mx1.freebsd.org (Postfix) with ESMTP id C55978FC18 for ; Mon, 3 Sep 2012 21:57:09 +0000 (UTC) Received: from server.rulingia.com (c220-239-249-137.belrs5.nsw.optusnet.com.au [220.239.249.137]) by vps.rulingia.com (8.14.5/8.14.5) with ESMTP id q83Lv9kE035401 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 4 Sep 2012 07:57:09 +1000 (EST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.14.5/8.14.5) with ESMTP id q83Lv3bl052886 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 4 Sep 2012 07:57:03 +1000 (EST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.14.5/8.14.5/Submit) id q83Lv38s052885; Tue, 4 Sep 2012 07:57:03 +1000 (EST) (envelope-from peter) Date: Tue, 4 Sep 2012 07:57:03 +1000 From: Peter Jeremy To: Ragnar Lonn Message-ID: <20120903215703.GB52619@server.rulingia.com> References: <50446681.2080307@gatorhole.com> <20120903210450.GC2654@aspire.rulingia.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="fdj2RfSjLxBAspz7" Content-Disposition: inline In-Reply-To: <20120903210450.GC2654@aspire.rulingia.com> X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hardware@freebsd.org Subject: Re: Load testing knocks out network X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 21:57:11 -0000 --fdj2RfSjLxBAspz7 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2012-Sep-04 07:04:50 +1000, Peter Jeremy wrote: >you will probably want to be running amd64 anyway. That said, Rod >Grimes was achieving between 100K and 1M TCP connections to FreeBSD >i386 systems in the 1990's. Oops, I misremembered. It was Terry Lambert achieving 1.6M connections, not Rod Grimes and only about 10 years ago: http://www.mavetju.org/mail/view_message.php?list=3Dfreebsd-hackers&id=3D15= 02550 (though I can't find when he started the work). --=20 Peter Jeremy --fdj2RfSjLxBAspz7 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlBFJ68ACgkQ/opHv/APuIf5GQCfcZkOJOUF89NkT/34AC366Lsu I3YAn3jyLnaJwhtV0pCsKAI8pg3HZPAM =UHKT -----END PGP SIGNATURE----- --fdj2RfSjLxBAspz7-- From owner-freebsd-hardware@FreeBSD.ORG Tue Sep 4 00:56:22 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD46B106564A for ; Tue, 4 Sep 2012 00:56:22 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 8CC068FC1A for ; Tue, 4 Sep 2012 00:56:21 +0000 (UTC) Received: by pbbrp2 with SMTP id rp2so8587728pbb.13 for ; Mon, 03 Sep 2012 17:56:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=BRh8bIQ6HcH7yYxoYh6Vn4BOAIAWjMTQTEQ3TN2G9kU=; b=NpEAIGu2Dh/Qga1JxVCqdA9Im5MnG/jJfXoJ6Lfzg2vqS+bhDjuEIGtpuAGaKar3Hs fOXgCk2GpwhwXO8Ebaqg6Vdz+kNSMzgayXeZECkHNHzw9tNRJPyRCihyDhCo2upnP4Ag sxmDZBIGl/HBkfqBuhgxVCd6zlGOkJD3itv845kBNPShyFFlFMnIynaQOq6LAws40Z6N BXTBbjCIouAevdRV97v/hCw2DiZvMJU8qAcgUdPh0r2QAx3rOjjbcZagjyYuIwhHODO6 c3IvR25SBJqEA9PZQKeipV6yB3GWgNUiWxtRtvjSuek55frZjGfQx/7IEzyrKhCNGVEB orJw== Received: by 10.66.76.130 with SMTP id k2mr37446504paw.19.1346720181387; Mon, 03 Sep 2012 17:56:21 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPS id hf1sm7397675pbc.42.2012.09.03.17.56.18 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 03 Sep 2012 17:56:20 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 04 Sep 2012 09:56:13 -0700 From: YongHyeon PYUN Date: Tue, 4 Sep 2012 09:56:13 -0700 To: Derrick Edwards Message-ID: <20120904165613.GB3126@michelle.cdnetworks.com> References: <201209030736.18334.dantavious313@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201209030736.18334.dantavious313@gmail.com> User-Agent: Mutt/1.4.2.3i Cc: freebsd-hardware@freebsd.org Subject: Re: Atheros AR8162/8166/8168 PCI-E Fast Ethernet Support X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 00:56:22 -0000 On Mon, Sep 03, 2012 at 07:36:13AM -0400, Derrick Edwards wrote: > Hi, > I purchased a Toshiba Satellite P875-S7200. I can not use the onboard ethernet > port because there is not a driver associated with it. I was wondering if an > driver was forthcoming supporting the subject chipsets. Also, are there any > work arounds. I was told that Qualcomm Atheros is working on writing driver for FreeBSD so it would be available in future. I don't know ETA though. > > FreeBSD 10.0-CURRENT #1: Sun Sep 2 06:02:47 EDT 2012 > > > none2@pci0:1:0:0: class=0x020000 card=0xff1e1179 chip=0x10901969 > rev=0x10 hdr=0x00 > vendor = 'Atheros Communications' > class = network > subclass = ethernet > bar [10] = type Memory, range 64, base 0xb8600000, size 262144, enabled > bar [18] = type I/O Port, range 32, base 0x2000, size 128, enabled > cap 01[40] = powerspec 3 supports D0 D3 current D0 > cap 10[58] = PCI-Express 1 endpoint max data 128(4096) link x1(x1) > cap 05[c0] = MSI supports 16 messages, 64 bit, vector masks > cap 11[d8] = MSI-X supports 16 messages in map 0x10 > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected > ecap 0003[180] = Serial 1 ff262ff500266cff > > V/r > Derrick From owner-freebsd-hardware@FreeBSD.ORG Tue Sep 4 02:47:58 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 705771065687 for ; Tue, 4 Sep 2012 02:47:58 +0000 (UTC) (envelope-from 33mtFUAgHEFMIJ_vFy39MNR.x970CzzwDy-2vCyHvCz0CzzwDy.9C1@photos-server.bounces.google.com) Received: from mail-pz0-f74.google.com (mail-pz0-f74.google.com [209.85.210.74]) by mx1.freebsd.org (Postfix) with ESMTP id 484858FC1E for ; Tue, 4 Sep 2012 02:47:58 +0000 (UTC) Received: by danp8 with SMTP id p8so459013dan.1 for ; Mon, 03 Sep 2012 19:47:58 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.74.69 with SMTP id r5mt8622454pav.12.1346726878106; Mon, 03 Sep 2012 19:47:58 -0700 (PDT) Message-ID: Date: Tue, 04 Sep 2012 02:47:58 +0000 From: xingyao wu To: freebsd-hardware@freebsd.org Content-Type: multipart/mixed; boundary=f46d042f931c2e18e404c8d7494a X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: xingyao wu shared photos with you X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: xingyao wu List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 02:47:58 -0000 --f46d042f931c2e18e404c8d7494a Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes Hello my Friend How are you? We have JBL speakers, YAMAHA amplifier,SHURE microphones, DBX equalizer. . . If you are interested in our products, please contact us, we can send the price list to you, we can give you the lowest price, best quality. We will make you satisfied. Thank you I wish you happy every day! Meiling --f46d042f931c2e18e404c8d7494a-- From owner-freebsd-hardware@FreeBSD.ORG Tue Sep 4 09:26:57 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CE7D1106564A for ; Tue, 4 Sep 2012 09:26:57 +0000 (UTC) (envelope-from dantavious313@gmail.com) Received: from mail-yw0-f45.google.com (mail-yw0-f45.google.com [209.85.213.45]) by mx1.freebsd.org (Postfix) with ESMTP id 848018FC0A for ; Tue, 4 Sep 2012 09:26:57 +0000 (UTC) Received: by yhpp34 with SMTP id p34so1081987yhp.18 for ; Tue, 04 Sep 2012 02:26:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:subject:date:user-agent:cc:references:in-reply-to :mime-version:content-type:content-transfer-encoding:message-id; bh=8eQdratSfB2lgBCG0RYqg4OgTe2isB2Xdysis58iI04=; b=e/5IMytHnqhx8iiWDM3UihrgJ5hQWg3BbukYlp381nd6k8V/mkCRvcUy1eBG0/8qoa 0dhBRdT+QXlNI3taodXXLtzeGHwtYfWYRgO68iHv5Heu6SQ5OI2REt9XaayfrHnVe8tg G4emgCm3ZYkSswvTNQZOdAdWyrdFrOtx8/+xJdi8lw5I9DIyulG6XedxRjQohaSSixo3 zCsK3cQPva4GCpJF4OUR8DaFONE/xEimIiOlhpagrmlGUBhbQR/Aw3S9CDwc/pl3pGeF h/65ZHJOjZJVpHXUs9m5I78xyToPTn0lwj8v+O8vSd7iofZCNPBHcN1WBJyd7SeCPS0Q ClZw== Received: by 10.236.79.97 with SMTP id h61mr16975104yhe.16.1346750811576; Tue, 04 Sep 2012 02:26:51 -0700 (PDT) Received: from joker.localnet (c-71-226-137-213.hsd1.ga.comcast.net. [71.226.137.213]) by mx.google.com with ESMTPS id p21sm28816310yhj.11.2012.09.04.02.26.50 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 04 Sep 2012 02:26:51 -0700 (PDT) From: Derrick Edwards To: pyunyh@gmail.com Date: Tue, 4 Sep 2012 05:26:49 -0400 User-Agent: KMail/1.13.7 (FreeBSD/10.0-CURRENT; KDE/4.8.4; amd64; ; ) References: <201209030736.18334.dantavious313@gmail.com> <20120904165613.GB3126@michelle.cdnetworks.com> In-Reply-To: <20120904165613.GB3126@michelle.cdnetworks.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201209040526.49771.dantavious313@gmail.com> Cc: freebsd-hardware@freebsd.org Subject: Re: Atheros AR8162/8166/8168 PCI-E Fast Ethernet Support X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 09:26:57 -0000 On Tuesday, September 04, 2012 12:56:13 PM YongHyeon PYUN wrote: > On Mon, Sep 03, 2012 at 07:36:13AM -0400, Derrick Edwards wrote: > > Hi, > > > > I purchased a Toshiba Satellite P875-S7200. I can not use the onboard > > ethernet port because there is not a driver associated with it. I was > > wondering if an driver was forthcoming supporting the subject chipsets. > > Also, are there any work arounds. > > I was told that Qualcomm Atheros is working on writing driver for > FreeBSD so it would be available in future. I don't know ETA > though. > > > FreeBSD 10.0-CURRENT #1: Sun Sep 2 06:02:47 EDT 2012 > > > > > > none2@pci0:1:0:0: class=0x020000 card=0xff1e1179 chip=0x10901969 > > rev=0x10 hdr=0x00 > > > > vendor = 'Atheros Communications' > > class = network > > subclass = ethernet > > bar [10] = type Memory, range 64, base 0xb8600000, size 262144, > > enabled bar [18] = type I/O Port, range 32, base 0x2000, size 128, > > enabled cap 01[40] = powerspec 3 supports D0 D3 current D0 > > cap 10[58] = PCI-Express 1 endpoint max data 128(4096) link x1(x1) > > cap 05[c0] = MSI supports 16 messages, 64 bit, vector masks > > cap 11[d8] = MSI-X supports 16 messages in map 0x10 > > > > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected > > ecap 0003[180] = Serial 1 ff262ff500266cff > > > > V/r > > Derrick Thanks for the update. That is good news. From owner-freebsd-hardware@FreeBSD.ORG Tue Sep 4 09:37:27 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4F7D9106566C for ; Tue, 4 Sep 2012 09:37:27 +0000 (UTC) (envelope-from ragnar@gatorhole.com) Received: from maple.lonn.org (maple.lonn.org [109.228.153.253]) by mx1.freebsd.org (Postfix) with ESMTP id E0D1D8FC20 for ; Tue, 4 Sep 2012 09:37:26 +0000 (UTC) Received: from [172.22.21.54] (unknown [82.99.15.130]) (Authenticated sender: ragnar@gatorhole.com) by maple.lonn.org (Postfix) with ESMTPSA id E7B2F735C52; Tue, 4 Sep 2012 11:37:19 +0200 (CEST) Message-ID: <5045CBCB.6000103@gatorhole.com> Date: Tue, 04 Sep 2012 11:37:15 +0200 From: Ragnar Lonn User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120827 Thunderbird/15.0 MIME-Version: 1.0 To: Peter Jeremy References: <50446681.2080307@gatorhole.com> <20120903210450.GC2654@aspire.rulingia.com> In-Reply-To: <20120903210450.GC2654@aspire.rulingia.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hardware@freebsd.org Subject: Re: Load testing knocks out network X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 09:37:27 -0000 On 09/03/2012 11:04 PM, Peter Jeremy wrote: > On 2012-Sep-03 10:12:49 +0200, Ragnar Lonn wrote: >> transmit/receive buffers. In newer *Linux* kernels, this memory is being >> allocated in an adaptive manner - i.e. the kernel only allocates a small >> amount of memory to each TCP buffer, and then increases it as necessary >> (per connection, depending on transfer speed and network delay to the >> other peer). > FreeBSD does this as well, though I don't recall when this was added. > >> I think I actually discussed this with FreeBSD developers a while ago >> (on this list even?), and they told me the FreeBSD kernel can only >> allocate max 2GB of kernel memory. > This is only true on 32-bit kernels. FreeBSD uses a single address > space so both kernel and userland need to fit into 4GB on 32-bit > systems. On 64-bit systems, KVM is less constrained (it's ~550GB on > my amd64). You can check sysctl's vm.kvm_free and vm.kvm_size for > exact figures. Maybe I misremembered slightly. I found the old discussion I had with people about this on the FreeBSD virtualization mailing list: http://osdir.com/ml/freebsd-virtualization/2009-02/msg00006.html Anyway, 1.6M connections sounds really good (although he only had 4GB of memory, so I guess the exercise was mostly academic - i.e. those connections would not be very useful in a real setting because each would have so little buffer memory). /Ragnar > >> 100K buffer memory. If you have e.g. 1GB available to network buffers, >> it means a max limit of 10k simultaneous connections on a server, >> regardless of how much memory it has. > If you want a system to usefully cope with 10K network connections, > you will probably want to be running amd64 anyway. That said, Rod > Grimes was achieving between 100K and 1M TCP connections to FreeBSD > i386 systems in the 1990's. > From owner-freebsd-hardware@FreeBSD.ORG Tue Sep 4 19:13:01 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 276EE106566B for ; Tue, 4 Sep 2012 19:13:01 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id F221B8FC1D for ; Tue, 4 Sep 2012 19:13:00 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 46CC4B999; Tue, 4 Sep 2012 15:13:00 -0400 (EDT) From: John Baldwin To: freebsd-hardware@freebsd.org Date: Tue, 4 Sep 2012 12:13:10 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p17; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201209041213.10931.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 04 Sep 2012 15:13:00 -0400 (EDT) Cc: Andy Young Subject: Re: What is rx_processing_limit sysctl for Intel igb NIC driver? X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 19:13:01 -0000 On Sunday, September 02, 2012 10:41:15 pm Andy Young wrote: > I am tuning our server that has an Intel 82576 gigabit NIC using the igb > driver. I see a lot of posts on the net where people bump the > rx_processing_limit sysctl from the default value of 100 to 4096. Can > anyone tell me what this is intended to do? If you have multiple devices sharing an IRQ with igb (and thus are not using MSI or MSI-X), it forces the driver to more-or-less cooperatively schedule with the other interrupts on the same IRQ. However, since igb uses a fast interrupt handler and a task on a dedicated taskqueue in the non-MSI case now, I think it doesn't even do that. It should probably be set to -1 (meaning unlimited) in just about all cases now. -- John Baldwin From owner-freebsd-hardware@FreeBSD.ORG Thu Sep 6 14:45:27 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 81B841065673; Thu, 6 Sep 2012 14:45:27 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id C971B8FC0C; Thu, 6 Sep 2012 14:45:26 +0000 (UTC) Received: by bkcje9 with SMTP id je9so935360bkc.13 for ; Thu, 06 Sep 2012 07:45:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=P6QGeliBiLXTWwSFVM0KLuo1S7slOMaAfLAB+2Hp8JA=; b=BRSOYKhO04obfCk6bo0clDPqfUdpbn047diAosZslhE8wbB75jWe9R+xWf8PoRB+zY 5PhIlxGMPcS5uN6nEsiAOIR8kVHxt9eySlZyEgiusHOaVC3yvbFiTEUJRhCiYdRG0i7E MeWVO99B4g0SFxc6zBDnHM6lYFgNXeSau4K8UtPKwIcn1b3UkOCsswe12zUj1jIAAGGP XekMYWwFfkwW+r8TRLIsDgyPs2/8pcIlSNhZOq/OS1XM10Ui5qzcE/+U6KAclr1vKDb3 7puYvJON+IB4saQxl6nmPI5kTYxmqs56oREFIYFFMvWmByklipRZEKjFhOmm0brGPNVs 0XDg== Received: by 10.204.13.82 with SMTP id b18mr1131544bka.118.1346942725455; Thu, 06 Sep 2012 07:45:25 -0700 (PDT) Received: from [10.0.0.86] ([93.152.184.10]) by mx.google.com with ESMTPS id y20sm1486405bkv.11.2012.09.06.07.45.23 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 06 Sep 2012 07:45:24 -0700 (PDT) Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) Content-Type: text/plain; charset=iso-8859-1 From: Nikolay Denev In-Reply-To: <201209041213.10931.jhb@freebsd.org> Date: Thu, 6 Sep 2012 17:45:21 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: <8876AEF9-0D0E-42A1-9B83-F2F7D36D7B7F@gmail.com> References: <201209041213.10931.jhb@freebsd.org> To: John Baldwin X-Mailer: Apple Mail (2.1486) Cc: Andy Young , freebsd-hardware@freebsd.org Subject: Re: What is rx_processing_limit sysctl for Intel igb NIC driver? X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 14:45:27 -0000 On Sep 4, 2012, at 7:13 PM, John Baldwin wrote: > On Sunday, September 02, 2012 10:41:15 pm Andy Young wrote: >> I am tuning our server that has an Intel 82576 gigabit NIC using the = igb >> driver. I see a lot of posts on the net where people bump the >> rx_processing_limit sysctl from the default value of 100 to 4096. Can >> anyone tell me what this is intended to do? >=20 > If you have multiple devices sharing an IRQ with igb (and thus are not = using=20 > MSI or MSI-X), it forces the driver to more-or-less cooperatively = schedule=20 > with the other interrupts on the same IRQ. However, since igb uses a = fast=20 > interrupt handler and a task on a dedicated taskqueue in the non-MSI = case now,=20 > I think it doesn't even do that. It should probably be set to -1 = (meaning > unlimited) in just about all cases now. >=20 > --=20 > John Baldwin > _______________________________________________ > freebsd-hardware@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hardware > To unsubscribe, send any mail to = "freebsd-hardware-unsubscribe@freebsd.org" And setting it to -1 gave a nice performance improvement in some tests = that I did recently. AFAIR only after setting this to -1 I was able to reach 10gig speed = using iperf on two directly connected machines with ix(4) 82599= From owner-freebsd-hardware@FreeBSD.ORG Fri Sep 7 14:53:04 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4A4AA1065670 for ; Fri, 7 Sep 2012 14:53:04 +0000 (UTC) (envelope-from ayoung@mosaicarchive.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id A582B8FC16 for ; Fri, 7 Sep 2012 14:53:03 +0000 (UTC) Received: by obbun3 with SMTP id un3so5989635obb.13 for ; Fri, 07 Sep 2012 07:53:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:date:message-id:subject:from:to :content-type:x-gm-message-state; bh=6ftCRvl4Bl0wO7hi7ljIt2nFQ91X25ei6SXwfarfITY=; b=pVaVJ2PHU/rsxuGgy09M0p/6LumQzmMUCkB9TRTzOzJzztDlws6TC6wkXyo2WSQqOB n06+IrofIFQGtdzwGvGziSh8+vEipaFgGaLupaSdVtOMbqzBVVOgsNKaKkG/fZFGmPBq E9+YWD0R5qAXGUdELlGySAPsXZ3LiEqaOnRamzQw3wMub5WJ4FEmsZGbWCOCTnlSBZ/G +FWD8gUXmpDzRPXCodX465ZEmcTOOwd1Gr1ymP9N5c6qSvd2S2ovG+05knZyq1QGZJ2j B0Bd2FI5A+B4z01yf0BxqRLhaTmB13gVvLYlzR6yLC6VtUELKhTe1dVqibLNY/ilydkZ uqfg== MIME-Version: 1.0 Received: by 10.182.188.41 with SMTP id fx9mr6156884obc.92.1347029582634; Fri, 07 Sep 2012 07:53:02 -0700 (PDT) Received: by 10.76.174.38 with HTTP; Fri, 7 Sep 2012 07:53:02 -0700 (PDT) X-Originating-IP: [75.147.53.134] Date: Fri, 7 Sep 2012 10:53:02 -0400 Message-ID: From: Andy Young To: freebsd-hardware@freebsd.org X-Gm-Message-State: ALoCoQmraO0AdSMypFYNefV7X4VS2Pt4DWKQjN8lMmEX+3xyPyQ9QNVruDkG5vWsRIkiAUgAYgdu Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Can FreeBSD handle redundant SAS controllers? X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 14:53:04 -0000 I'm having trouble finding information about this on the web. I have a SAS backplane with redundant expanders designed so you can use redundant SAS controllers. Seems obvious to me that the drivers need to support two cards pointing to the same drives. Does this work in FreeBSD? Does it matter which HBAs I buy? Thanks! Andy From owner-freebsd-hardware@FreeBSD.ORG Sat Sep 8 19:34:28 2012 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1C51A106566C for ; Sat, 8 Sep 2012 19:34:28 +0000 (UTC) (envelope-from pschmehl_lists@tx.rr.com) Received: from cdptpa-omtalb.mail.rr.com (cdptpa-omtalb.mail.rr.com [75.180.132.120]) by mx1.freebsd.org (Postfix) with ESMTP id D6EF38FC08 for ; Sat, 8 Sep 2012 19:34:27 +0000 (UTC) X-Authority-Analysis: v=2.0 cv=H+ZZMpki c=1 sm=0 a=+L5dYfeubEW4PLvjDgtIXQ==:17 a=WAZfUmVf-EkA:10 a=05ChyHeVI94A:10 a=kj9zAlcOel0A:10 a=ayC55rCoAAAA:8 a=IVZR23WZ1MQA:10 a=UdvEN9gbqWsBAELrqE0A:9 a=CjuIK1q_8ugA:10 a=+L5dYfeubEW4PLvjDgtIXQ==:117 X-Cloudmark-Score: 0 X-Originating-IP: 76.184.157.127 Received: from [76.184.157.127] ([76.184.157.127:54018] helo=[10.0.0.133]) by cdptpa-oedge02.mail.rr.com (envelope-from ) (ecelerity 2.2.3.46 r()) with ESMTP id AC/FF-15382-7BD9B405; Sat, 08 Sep 2012 19:34:21 +0000 Date: Sat, 08 Sep 2012 14:34:14 -0500 From: Paul Schmehl To: FreeBSD Hardware Message-ID: X-Mailer: Mulberry/4.0.8 (Mac OS X) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: Dell Perc H200A X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Paul Schmehl List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2012 19:34:28 -0000 I tried to install FreeBSD 9.0 Release on a Dell R410 with Perc H200A today. When I tried ran the new installer it failed. So I switched to sysinstall, but it couldn't recognize the disk. I googled and found out that there's been a problem with no driver for that controller. If I understand it correctly, the mps driver works, but you had to use something called JBOD mode. I have no idea what that is or how to set it up. I'm not real knowledgeable about RAID and controllers and the like. All I want to do is install FreeBSD on this box. Is there a version of FreeBSD that will work with the Perc H200A? If so, how do I get it? Paul Schmehl, Senior Infosec Analyst As if it wasn't already obvious, my opinions are my own and not those of my employer. ******************************************* "It is as useless to argue with those who have renounced the use of reason as to administer medication to the dead." Thomas Jefferson "There are some ideas so wrong that only a very intelligent person could believe in them." George Orwell