Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 5 Jul 2002 00:20:56 -0400
From:      Bosko Milekic <bmilekic@unixdaemons.com>
To:        Andrew Gallatin <gallatin@cs.duke.edu>
Cc:        "Kenneth D. Merry" <ken@kdm.org>, current@FreeBSD.ORG, net@FreeBSD.ORG
Subject:   Re: virtually contig jumbo mbufs (was Re: new zero copy sockets snapshot)
Message-ID:  <20020705002056.A5365@unixdaemons.com>
In-Reply-To: <15652.46870.463359.853754@grasshopper.cs.duke.edu>; from gallatin@cs.duke.edu on Thu, Jul 04, 2002 at 04:59:02PM -0400
References:  <xzpelf3ida1.fsf@flood.ping.uio.no> <20020619090046.A2063@panzer.kdm.org> <20020619120641.A18434@unixdaemons.com> <15633.17238.109126.952673@grasshopper.cs.duke.edu> <20020619233721.A30669@unixdaemons.com> <15633.62357.79381.405511@grasshopper.cs.duke.edu> <20020620114511.A22413@unixdaemons.com> <15634.534.696063.241224@grasshopper.cs.duke.edu> <20020620134723.A22954@unixdaemons.com> <15652.46870.463359.853754@grasshopper.cs.duke.edu>

next in thread | previous in thread | raw e-mail | index | archive | help

On Thu, Jul 04, 2002 at 04:59:02PM -0400, Andrew Gallatin wrote:
>  >   I believe that the Intel chips do "virtual page caching" and that the
>  > logic that does the virtual -> physical address translation sits between
>  > the L2 cache and RAM.  If that is indeed the case, then your idea of
>  > testing with virtually contiguous pages is a good one.
>  >   Unfortunately, I don't know if the PIII is doing speculative
>  > cache-loads, but it could very well be the case.  If it is and if in
>  > fact the chip does caching based on virtual addresses, then providing it
>  > with virtually contiguous address space may yield better results.  If
>  > you try this, please let me know.  I'm extremely interested in seeing
>  > the results!
> 
> contigmalloc'ed private jumbo mbufs (same as bge, if_ti, etc):
> 
> % iperf -c ugly-my -l 32k -fm
> ------------------------------------------------------------
> Client connecting to ugly-my, TCP port 5001
> TCP window size:  0.2 MByte (default)
> ------------------------------------------------------------
> [  3] local 192.168.1.3 port 1031 connected with 192.168.1.4 port 5001
> [ ID] Interval       Transfer     Bandwidth
> [  3]  0.0-10.0 sec  2137 MBytes  1792 Mbits/sec
> 
> 
> 
> malloc'ed, physically discontigous private jumbo mbufs:
> 
> % iperf -c ugly-my -l 32k -fm
> ------------------------------------------------------------
> Client connecting to ugly-my, TCP port 5001
> TCP window size:  0.2 MByte (default)
> ------------------------------------------------------------
> [  3] local 192.168.1.3 port 1029 connected with 192.168.1.4 port 5001
> [ ID] Interval       Transfer     Bandwidth
> [  3]  0.0-10.0 sec  2131 MBytes  1788 Mbits/sec
> 
> 
> So I'd be willing to believe that the 4Mb/sec loss was due to 
> the extra overhead of setting up 2 additional DMAs. 
> 
> 
> So it looks like this idea would work. 

 Yes, it certainly confirms the virtual-based caching assumptions.  I
 would like to provide virtually contiguous large buffers and believe I
 can do that via mb_alloc... however, they would be several wired-down
 pages.  Would this be in line with the requirements that these buffers
 would have, in your mind?  (wired-down means that your buffers will
 come out exactly as they would out of malloc(), so if you were using
 malloc() already, I'm assuming that wired-down is OK).

 I think I can allocate the jumbo buffers via mb_alloc from the same map
 as I allocate clusters from - the clust_map - and keep them in
 buckets/slabs in per-CPU caches, like I do for mbufs and regular
 clusters right now.  Luigi is in the process of doing some optimisation
 work around mb_alloc and I'll probably be doing the SMP-specific stuff
 after he's done so once that's taken care of, we can take a stab at
 this if you think it's worth it.

> Drew
 
Regards,
-- 
Bosko Milekic
bmilekic@unixdaemons.com
bmilekic@FreeBSD.org


To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-net" in the body of the message




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020705002056.A5365>