From owner-freebsd-net@freebsd.org Tue Aug 25 14:45:53 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2F38E99D4AE for ; Tue, 25 Aug 2015 14:45:53 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 09E87691 for ; Tue, 25 Aug 2015 14:45:52 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (ppp121-45-243-143.lns20.per4.internode.on.net [121.45.243.143]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id t7PEjkBB005480 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Tue, 25 Aug 2015 07:45:49 -0700 (PDT) (envelope-from julian@freebsd.org) Subject: Re: FreeBSD 10.2 , ospf vs. aggregated static routes, performance issue To: Evgeny Khorokhorin , freebsd-net@freebsd.org References: <55DC76B7.9060606@maxnet.ru> From: Julian Elischer Message-ID: <55DC7F94.1080908@freebsd.org> Date: Tue, 25 Aug 2015 22:45:40 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <55DC76B7.9060606@maxnet.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Aug 2015 14:45:53 -0000 On 8/25/15 10:07 PM, Evgeny Khorokhorin wrote: > Hi, > > I have 10.2-STABLE, 2 CPU Intel E5-2643v3, network Intel XL710 with > 1.4.0 driver from Intel > I know that going through routing table is very fast (rn_match). But > I decided to optimize routing table. > I'm using 2 interfaces - ixl0 and ixl1. > Behind ixl0 I have 304 networks 172.16.. from /28 to /24 all via the > same gw 1.1.1.1 (because ip on ixl0 with /30 mask). And behind ixl1 > I have default route via 2.2.2.2. > That 304 172.16 networks I receive via OSPF (quagga). Now all is OK > - on every interface I have up to 500kpps/395kpps, 4.5Gbps/1.57Gbps > (rx/tx on ixl1 and tx/rx on ixl0). > If I disable OSPF and in zebra add static route 172.16.0.0/12 via > 1.1.1.1, the system works good until traffic grow up to > 251kpps/181kpps , 2.27Gbps/637Mbps. After that the system is > degrading: ixl's queue threads utilizes 100% CPU and I see many many > traffic drops (netstat -i) > If I turn on ospfd and receive 304 more specific routes the problem > disappears. > > Where is the problem? Or I have misunderstanding about how FreeBSD > uses routing table.. > P.S. I use this machine as NAT. I checked this on ipfw and pf, all > the same. without knowing anything, it looks like a lock on the route is a bottleneck. lots of routes spreads the pain.. try two manually added static routes 172.16.0.0/13 and 172.24.0.0/13 (I hope I split that correctly) and see if it changes things.. then try 4.. > > -- Cheers, > Evgeny > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >