Date: Fri, 11 Jun 2010 11:21:24 -0600 From: Scott Long <scottl@samsco.org> To: Marcel Moolenaar <xcllnt@mac.com> Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, John Baldwin <jhb@freebsd.org> Subject: Re: svn commit: r209026 - in head/sys/ia64: ia64 include Message-ID: <EE4B5C07-194D-4408-B0A1-BA25BE39468F@samsco.org> In-Reply-To: <19B0DF11-5998-40F5-8095-8D2521B1C597@mac.com> References: <201006110300.o5B30X9q045387@svn.freebsd.org> <201006110751.40735.jhb@freebsd.org> <853068F6-D736-4DA3-859F-D946D096843D@samsco.org> <19B0DF11-5998-40F5-8095-8D2521B1C597@mac.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Jun 11, 2010, at 11:04 AM, Marcel Moolenaar wrote: >=20 > On Jun 11, 2010, at 9:12 AM, Scott Long wrote: >=20 >> On Jun 11, 2010, at 5:51 AM, John Baldwin wrote: >>> On Thursday 10 June 2010 11:00:33 pm Marcel Moolenaar wrote: >>>> Author: marcel >>>> Date: Fri Jun 11 03:00:32 2010 >>>> New Revision: 209026 >>>> URL: http://svn.freebsd.org/changeset/base/209026 >>>>=20 >>>> Log: >>>> Bump MAX_BPAGES from 256 to 1024. It seems that a few drivers, = bge(4) >>>> in particular, do not handle deferred DMA map load operations at = all. >>>> Any error, and especially EINPROGRESS, is treated as a hard error = and >>>> typically abort the current operation. The fact that the busdma = code >>>> queues the load operation for when resources (i.e. bounce buffers = in >>>> this particular case) are available makes this especially = problematic. >>>> Bounce buffering, unlike what the PR synopsis would suggest, works >>>> fine. >>>>=20 >>>> While on the subject, properly implement swi_vm(). >>>=20 >>> NIC drivers do not handle deferred load operations at all (note that=20= >>> bus_dmamap_load_mbuf() and bus_dmamap_load_mbuf_sg() enforce = BUS_DMA_NOWAIT). >>> It is common practice to just drop the packet in that case. >>>=20 >>=20 >> Yes, long ago when network drivers started being converted to busdma, = it was agreed that EINPROGRESS simply doesn't make sense for them. Any = platform that winds up making extensive use of bounce buffers for = network hardware is going to perform poorly no matter what, and should = hopefully have some sort of IOMMU that can be used instead. >=20 > Unfortunately things aren't as simple as is presented. >=20 > For one, bge(4) wedges as soon as the platform runs out of bounce > buffers when they're needed. The box needs to be reset in order to > get the interface back. I pick any implementation that remains > functional over a mis-optimized one that breaks. Deferred load > operations are more performance optimal than failure is. >=20 This sounds like a bug in the bge driver. I don't see if through casual = inspection, but the driver should be able to either drop the mbuf = entirely, or requeue it on the ifq and then restart the ifq later. > Also: the kernel does nothing to guarantee maximum availability > of DMA-able memory under load, so bounce buffers (or use of I/O > MMUs for that matter) are a reality. Here too the performance > argument doesn't necessarily hold because the kernel may be > busy with more than just sending and receiving packets and the > need to defer load operations is very appropriate. If the > alternative is just dropped packets, I'm fine with that too, > but I for one cannot say that *not* filling a H/W ring with > buffers is not going to wedge the hardware in some cases. >=20 > Plus: SGI Altix does not have any DMA-able memory for 32-bit > hardware. The need for an I/O MMU is absolute and since there > are typically less mapping registers than packets on a ring, > the need for deferred operation seems quite acceptable if the > alternative is, again, failure to operate. >=20 I'm not against you upping the bounce buffer limit for a particular = platform, but it's still unclear to me if (given bug-free drivers) it's = worth the effort to defer a load rather than just drop the packet and = let the stack retry it. One question that would be good to answer is = wether the failed load is happening in the RX to TX path. Scott
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?EE4B5C07-194D-4408-B0A1-BA25BE39468F>