From owner-freebsd-net@FreeBSD.ORG Sat Dec 17 16:53:05 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD1551065673 for ; Sat, 17 Dec 2011 16:53:05 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 6F5AB8FC18 for ; Sat, 17 Dec 2011 16:53:05 +0000 (UTC) Received: by iadj38 with SMTP id j38so3050480iad.13 for ; Sat, 17 Dec 2011 08:53:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=hxcoHe/ZYXd7etizaB0RpjjQyKGtkV0bhVJW/LkXzXk=; b=i2XzZmBsKDv6coGg/qDkL47V9KT6uEubRpAF/zIH7MLqMTHxz4mFf5eGN3ZBY8SGPC mO/UhyJ6JCZlFtoc7QS6uBNd8gBGfX3eJYeMQ+LiilCPnmVtkEVAaRiv25Y/X9OD2TOJ j0MhCSGnFIZO++GOD4ouRwbl+PFCw7b3sr0Fw= MIME-Version: 1.0 Received: by 10.50.94.170 with SMTP id dd10mr16046613igb.41.1324140784742; Sat, 17 Dec 2011 08:53:04 -0800 (PST) Received: by 10.231.41.206 with HTTP; Sat, 17 Dec 2011 08:53:04 -0800 (PST) In-Reply-To: References: <20111205022730.GE1729@michelle.cdnetworks.com> <20111207005958.GE5825@michelle.cdnetworks.com> <20111207235810.GB9211@michelle.cdnetworks.com> <20111208215256.GB13605@michelle.cdnetworks.com> Date: Sat, 17 Dec 2011 18:53:04 +0200 Message-ID: From: Sami Halabi To: Arnaud Lacombe Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: pyunyh@gmail.com, freebsd-net@freebsd.org Subject: Re: Problem on re0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2011 16:53:05 -0000 Hi, here is # pciconf -lv | grep -A4 bge\[0-9]\@ bge0@pci0:2:0:0: class=0x020000 card=0x03781014 chip=0x165a14e4 rev=0x00 hdr=0x00 vendor = 'Broadcom Corporation' device = 'Broadcom NetXtreme BCM5722 Gigabit (94309)' class = network subclass = ethernet -- bge1@pci0:3:1:0: class=0x020000 card=0x026f1014 chip=0x16c714e4 rev=0x10 hdr=0x00 vendor = 'Broadcom Corporation' device = 'BCM5703A3 NetXtreme Gigabit Ethernet' class = network subclass = ethernet and dmesg: bge1: mem 0xe8400000-0xe840ffff irq 21 at device 1.0 on pci3 bge1: CHIP ID 0x00001100; ASIC REV 0x01; CHIP REV 0x11; PCI miibus1: on bge1 brgphy1: PHY 1 on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge1: Ethernet address: 00:1a:64:21:86:29 bge1: [ITHREAD] Thanks On Sat, Dec 17, 2011 at 6:49 PM, Sami Halabi wrote: > Hi, > > I've moved to a new server with bge card driver, similar configuration: > /etc/sysctl.conf > net.inet.flowtable.enable=0 > net.inet.ip.fastforwarding=1 > kern.ipc.somaxconn=8192 > kern.ipc.shmmax=2147483648 > kern.ipc.maxsockets=204800 > kern.ipc.maxsockbuf=2097152 > hw.intr_storm_threshold=9000 > kern.maxfiles=256000 > kern.maxfilesperproc=230400 > net.inet.ip.dummynet.pipe_slot_limit=1000 > #net.inet.ip.dummynet.io_fast=1 > net.link.ether.ipfw=1 > kern.ipc.nmbclusters=409600 > net.graph.recvspace=40960 > net.graph.maxdgram=40960 > > Kernel > ----------- > device lagg > options IPFIREWALL > options IPFIREWALL_FORWARD > options IPFIREWALL_VERBOSE > options DUMMYNET > options HZ=1000 > options TCP_SIGNATURE > device crypto # core crypto support > device cryptodev # /dev/crypto for access to h/w > options IPSEC > options DEVICE_POLLING > device if_bridge > > /boot/loader.conf > net.graph.maxalloc=128000 > net.graph.maxdata=128000 > net.graph.threads=4 > > and i have similar problem.. i check each minute for ping and i see > sometimes loss of 50% or 100% (ping -c 2 -t 4 otherside.ip.com), > here are the sysctl dev.bge.1.stats: > dev.bge.1.stats.FramesDroppedDueToFilters: 0 > dev.bge.1.stats.DmaWriteQueueFull: 2291350 > dev.bge.1.stats.DmaWriteHighPriQueueFull: 0 > dev.bge.1.stats.NoMoreRxBDs: 0 > dev.bge.1.stats.InputDiscards: 0 > dev.bge.1.stats.InputErrors: 0 > dev.bge.1.stats.RecvThresholdHit: 124120300 > dev.bge.1.stats.DmaReadQueueFull: 13470948 > dev.bge.1.stats.DmaReadHighPriQueueFull: 962 > dev.bge.1.stats.SendDataCompQueueFull: 0 > dev.bge.1.stats.RingSetSendProdIndex: 349289174 > dev.bge.1.stats.RingStatusUpdate: 260304688 > dev.bge.1.stats.Interrupts: 260304688 > dev.bge.1.stats.AvoidedInterrupts: 0 > dev.bge.1.stats.SendThresholdHit: 0 > dev.bge.1.stats.rx.ifHCInOctets: 2165908673 > dev.bge.1.stats.rx.Fragments: 0 > dev.bge.1.stats.rx.UnicastPkts: 237503495 > dev.bge.1.stats.rx.MulticastPkts: 0 > dev.bge.1.stats.rx.FCSErrors: 0 > dev.bge.1.stats.rx.AlignmentErrors: 0 > dev.bge.1.stats.rx.xonPauseFramesReceived: 0 > dev.bge.1.stats.rx.xoffPauseFramesReceived: 0 > dev.bge.1.stats.rx.ControlFramesReceived: 0 > dev.bge.1.stats.rx.xoffStateEntered: 0 > dev.bge.1.stats.rx.FramesTooLong: 0 > dev.bge.1.stats.rx.Jabbers: 0 > dev.bge.1.stats.rx.UndersizePkts: 0 > dev.bge.1.stats.rx.inRangeLengthError: 0 > dev.bge.1.stats.rx.outRangeLengthError: 0 > dev.bge.1.stats.tx.ifHCOutOctets: 1956421618 > dev.bge.1.stats.tx.Collisions: 0 > dev.bge.1.stats.tx.XonSent: 0 > dev.bge.1.stats.tx.XoffSent: 0 > dev.bge.1.stats.tx.flowControlDone: 0 > dev.bge.1.stats.tx.InternalMacTransmitErrors: 0 > dev.bge.1.stats.tx.SingleCollisionFrames: 0 > dev.bge.1.stats.tx.MultipleCollisionFrames: 0 > dev.bge.1.stats.tx.DeferredTransmissions: 0 > dev.bge.1.stats.tx.ExcessiveCollisions: 0 > dev.bge.1.stats.tx.LateCollisions: 0 > dev.bge.1.stats.tx.UnicastPkts: 347260508 > dev.bge.1.stats.tx.MulticastPkts: 0 > dev.bge.1.stats.tx.BroadcastPkts: 30306 > dev.bge.1.stats.tx.CarrierSenseErrors: 0 > dev.bge.1.stats.tx.Discards: 0 > dev.bge.1.stats.tx.Errors: 0 > > this driver gives more statics... > > please help me, this causes us serious problems with customers. > > Thanks in advance, > Sami > > On Fri, Dec 9, 2011 at 8:51 AM, Arnaud Lacombe wrote: > >> Hi, >> >> On Thu, Dec 8, 2011 at 4:52 PM, YongHyeon PYUN wrote: >> > On Thu, Dec 08, 2011 at 11:56:51AM +0200, Sami Halabi wrote: >> >> hi, >> >> i can see what it does. >> >> how do i compile the driver and load it instead of the running driver >> >> without restarting? its a production server. >> > >> > There is no such way. You have to rebuild/install kernel and >> > reboot. >> > >> How so ? AFAIK, you should be able to build the driver as a module, >> tell the bootloader to pre-load the module so that the built-in driver >> ends up being ignored. I did that some time ago with em(4). From >> memory, the steps should be: >> >> # From the source root >> % cd sys/modules/re >> % make >> % scp if_re.ko :/boot/kernel >> >> then on the target, add in `/boot/loader.conf': >> >> if_re_load="YES" >> >> - Arnaud >> > > > > -- > Sami Halabi > Information Systems Engineer > NMS Projects Expert > > -- Sami Halabi Information Systems Engineer NMS Projects Expert