Date: Mon, 19 Dec 2011 08:54:07 +0200 From: Sami Halabi <sodynet1@gmail.com> To: pyunyh@gmail.com Cc: freebsd-net@freebsd.org, Arnaud Lacombe <lacombar@gmail.com> Subject: Re: Problem on re0 Message-ID: <CAEW%2BogbxGSVY4%2BJPxneeH25-gZNsM7%2BPFZRT2okHdpYJHvYgXg@mail.gmail.com> In-Reply-To: <20111219022624.GB1704@michelle.cdnetworks.com> References: <CAEW%2BogauicnjsNdMCBBtoYzOLKVQoxP92Lw9Q_YgYPAe96dS5g@mail.gmail.com> <20111205022730.GE1729@michelle.cdnetworks.com> <CAEW%2BogbPwPY8r=uHhpkkp9-_K_gNsAVzKhVmgWScAMsHt2SJKQ@mail.gmail.com> <20111207005958.GE5825@michelle.cdnetworks.com> <CAEW%2BogZ7wivLcNm8rpcDR-ekP7Z6iEBZ0DtExhg5NVGfho0Ckw@mail.gmail.com> <20111207235810.GB9211@michelle.cdnetworks.com> <CAEW%2BogZKetpmwuCuyQo-g5DVZ4jCFM0L-mRApV9RTQ4S=Yi3hg@mail.gmail.com> <20111208215256.GB13605@michelle.cdnetworks.com> <CACqU3MVyDD=Vj1wK%2BNmrjjauzq85JSqrj2WEjLj878fSqhzT%2BQ@mail.gmail.com> <CAEW%2BogZOyBDN36t0BMEFcgKgWydShuC82Ym5=eDUAGNCvU0w5Q@mail.gmail.com> <20111219022624.GB1704@michelle.cdnetworks.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, >It seems you have really complex network configuration(ipfw(4), >lagg(4), dummynet(4), bridge(4) and ipsec(4) etc). Finding out >simplest network setup that shows the issue would be required here. hmm.. i actually don't use lagg nor bridge nor ipsec, i compile them for any future use that might be. however i use IPFW, the MPD port limit connected users via pptp/l2tp with ipfw rules via netgraph (i must admit i don't see them when I use ipfw show) and probably dummynet. Archetecture is simple: bge0 -- connected to border router, bge1 connected to cables company where customers connect we are routing 172.16.0.0/12 to bge1, default gateway is our peer in the other side of bge0. installed port as pptp/l2tp server is MPD. please help me its really making us troubles... Sami On Mon, Dec 19, 2011 at 4:26 AM, YongHyeon PYUN <pyunyh@gmail.com> wrote: > On Sat, Dec 17, 2011 at 06:49:48PM +0200, Sami Halabi wrote: > > Hi, > > > > I've moved to a new server with bge card driver, similar configuration: > > /etc/sysctl.conf > > net.inet.flowtable.enable=0 > > net.inet.ip.fastforwarding=1 > > kern.ipc.somaxconn=8192 > > kern.ipc.shmmax=2147483648 > > kern.ipc.maxsockets=204800 > > kern.ipc.maxsockbuf=2097152 > > hw.intr_storm_threshold=9000 > > kern.maxfiles=256000 > > kern.maxfilesperproc=230400 > > net.inet.ip.dummynet.pipe_slot_limit=1000 > > #net.inet.ip.dummynet.io_fast=1 > > net.link.ether.ipfw=1 > > kern.ipc.nmbclusters=409600 > > net.graph.recvspace=40960 > > net.graph.maxdgram=40960 > > > > Kernel > > ----------- > > device lagg > > options IPFIREWALL > > options IPFIREWALL_FORWARD > > options IPFIREWALL_VERBOSE > > options DUMMYNET > > options HZ=1000 > > options TCP_SIGNATURE > > device crypto # core crypto support > > device cryptodev # /dev/crypto for access to h/w > > options IPSEC > > options DEVICE_POLLING > > device if_bridge > > > > /boot/loader.conf > > net.graph.maxalloc=128000 > > net.graph.maxdata=128000 > > net.graph.threads=4 > > > > and i have similar problem.. i check each minute for ping and i see > > sometimes loss of 50% or 100% (ping -c 2 -t 4 otherside.ip.com), > > here are the sysctl dev.bge.1.stats: > > dev.bge.1.stats.FramesDroppedDueToFilters: 0 > > dev.bge.1.stats.DmaWriteQueueFull: 2291350 > > dev.bge.1.stats.DmaWriteHighPriQueueFull: 0 > > dev.bge.1.stats.NoMoreRxBDs: 0 > > dev.bge.1.stats.InputDiscards: 0 > > dev.bge.1.stats.InputErrors: 0 > > dev.bge.1.stats.RecvThresholdHit: 124120300 > > dev.bge.1.stats.DmaReadQueueFull: 13470948 > > dev.bge.1.stats.DmaReadHighPriQueueFull: 962 > > dev.bge.1.stats.SendDataCompQueueFull: 0 > > dev.bge.1.stats.RingSetSendProdIndex: 349289174 > > dev.bge.1.stats.RingStatusUpdate: 260304688 > > dev.bge.1.stats.Interrupts: 260304688 > > dev.bge.1.stats.AvoidedInterrupts: 0 > > dev.bge.1.stats.SendThresholdHit: 0 > > dev.bge.1.stats.rx.ifHCInOctets: 2165908673 > > dev.bge.1.stats.rx.Fragments: 0 > > dev.bge.1.stats.rx.UnicastPkts: 237503495 > > dev.bge.1.stats.rx.MulticastPkts: 0 > > dev.bge.1.stats.rx.FCSErrors: 0 > > dev.bge.1.stats.rx.AlignmentErrors: 0 > > dev.bge.1.stats.rx.xonPauseFramesReceived: 0 > > dev.bge.1.stats.rx.xoffPauseFramesReceived: 0 > > dev.bge.1.stats.rx.ControlFramesReceived: 0 > > dev.bge.1.stats.rx.xoffStateEntered: 0 > > dev.bge.1.stats.rx.FramesTooLong: 0 > > dev.bge.1.stats.rx.Jabbers: 0 > > dev.bge.1.stats.rx.UndersizePkts: 0 > > dev.bge.1.stats.rx.inRangeLengthError: 0 > > dev.bge.1.stats.rx.outRangeLengthError: 0 > > dev.bge.1.stats.tx.ifHCOutOctets: 1956421618 > > dev.bge.1.stats.tx.Collisions: 0 > > dev.bge.1.stats.tx.XonSent: 0 > > dev.bge.1.stats.tx.XoffSent: 0 > > dev.bge.1.stats.tx.flowControlDone: 0 > > dev.bge.1.stats.tx.InternalMacTransmitErrors: 0 > > dev.bge.1.stats.tx.SingleCollisionFrames: 0 > > dev.bge.1.stats.tx.MultipleCollisionFrames: 0 > > dev.bge.1.stats.tx.DeferredTransmissions: 0 > > dev.bge.1.stats.tx.ExcessiveCollisions: 0 > > dev.bge.1.stats.tx.LateCollisions: 0 > > dev.bge.1.stats.tx.UnicastPkts: 347260508 > > dev.bge.1.stats.tx.MulticastPkts: 0 > > dev.bge.1.stats.tx.BroadcastPkts: 30306 > > dev.bge.1.stats.tx.CarrierSenseErrors: 0 > > dev.bge.1.stats.tx.Discards: 0 > > dev.bge.1.stats.tx.Errors: 0 > > > > this driver gives more statics... > > > > please help me, this causes us serious problems with customers. > > > > Given that you see the same issue with bge(4) it looks like the > root cause is not in ethernet driver. I also see no evidence of > dropped frames from bge(4) hardware MAC statistics. > It seems you have really complex network configuration(ipfw(4), > lagg(4), dummynet(4), bridge(4) and ipsec(4) etc). Finding out > simplest network setup that shows the issue would be required here. > -- Sami Halabi Information Systems Engineer NMS Projects Expert
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAEW%2BogbxGSVY4%2BJPxneeH25-gZNsM7%2BPFZRT2okHdpYJHvYgXg>