Date: Fri, 07 Sep 2007 20:38:19 +0100 From: Gary Palmer <gpalmer@freebsd.org> To: Kirc Gover <kirc.gover@yahoo.com.au> Cc: freebsd-net@freebsd.org Subject: Re: OS choice for an edge router Message-ID: <46E1A8AB.7070708@freebsd.org> In-Reply-To: <517436.12027.qm@web44802.mail.sp1.yahoo.com> References: <517436.12027.qm@web44802.mail.sp1.yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Kirc Gover wrote: > Hi Gary, Thanks for your response. > > Yes, the bus architecture would be either PCI-X or PCI-Express. Host CP= U would be a high performance multi-core such as Xeon and NICs would be I= ntel. > > One of my concern is on the native forwarding capability of FreeBSD OS = and the execution of critical userland processes. I have experience befor= e that a FreeBSD box configured as router appears to slow down the userla= nd processes when the traffic load is high. I have verified this lately o= n 6.1, running on Athlon64 with 1G NIC cards with PF and ALTQ (queuing) e= nabled. I'm not so sure if this is caused by PF or ALTQ. Looking at the = processes using top, it could see that "swi(x) net" process is almost as = near 100% cpu utilization. At this state, the box can still forward traff= ic (not sure yet if the was a change in throughput) but I could notice th= e userland processes to be very slow, like invoking any command from the= shell (e.g. ls) will take so long to be executed and completed. Is this = a know limitation or bug? > =20 I'm probably not the best person to address that, but its my=20 understanding that interrupts will preempt other activity such as=20 userland processing. Since there is often a requirement to service=20 interrupts as a matter of priority (e.g. before the receive buffer in=20 the NIC overflows) I'm not sure theres a way around it. The performance = improvements in 7.x, especially relating to multi-core/CPU environments=20 might help. The complexity of the PF / ALTQ rules can also have an=20 impact, although I'm a little surprised that they counted towards=20 interrupt activity, which your message seems to imply. Gary P.S. Please don't top post. > Thanks a lot. > Kirc > > Gary Palmer <gpalmer@freebsd.org> wrote: On Fri, Sep 07, 2007 at 03:06:= 54AM +1000, Kirc Gover wrote: > =20 >> We are in the stage of planning and research for a commercial developm= ent of an edge router that will be based mostly on OpenSource software. I= would like to solicit for information and recommendation if FreeBSD is a= suitable OS. The router is expected to withstand forwarding of sustained= traffic from 10Mbps to 1Gbps and maybe more than that. Are there any kno= wn limitations of FreeBSD in terms of architecture and performance? Can I= just take out a FreeBSD as is and put it with the hardware without any s= pecific or major refinements in its code? I'm very much concerned with i= ts capability in forwarding heavy sustained traffic. Packet loss should b= e at minimum and critical userland processes should working normally eve= n under heavy load. Are there any known specific limitations of FreeBSD? = I have browsed through the archives and found a lot of hangups, deadlocks= and freeze issues. What is the usual or minimum hardware requirement? Is= soekris box enough, or dual core or ASIC >> based platforms? I'm aware that there are so many FreeBSD based route= rs and network based devices in the market. Is this a way to go over real= time and embedded OS such as VxWorks and others (mostly commercial) witho= ut putting the licensing cost in picture? I really appreciate any help, s= uggestions and recommendations. More power to FreeBSD! >> =20 > > Kirc, > > There are two factors to consider: > > - bus architecture (PCI, PCI-X, PCI-Express, etc) will dictate the > maximum throughput in bits/sec. Allow some overhead for bus arbitrat= ion > activities, and remember that the packet crosses the bus twice, once > on the inbound leg and once on the outbound leg. > > - Host CPU (and perhaps to a limited extent the interface cards used) > will dictate the packets per second (PPS) > > Most commercial routers run out of packets per second (in real-world > situations, not lab mockups) long before the theoretical maximum > throughput is achieved. Thinks like ICMP ping packets and TCP RST > packets are small (less than 100 bytes usually) but normally take as mu= ch > CPU to process & route as a 1500 byte (or larger) packet. > > The more you put in the processing path (e.g. packet filters, complex > routing tables) the more you reduce the PPS. > > Gary > > > =20 > --------------------------------- > Sick of deleting your inbox? Yahoo!7 Mail has free unlimited storage. G= et it now. > =20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?46E1A8AB.7070708>