Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 3 Feb 2006 17:47:32 -0500 (EST)
From:      Andrew Gallatin <gallatin@cs.duke.edu>
To:        freebsd-current@freebsd.org
Subject:   machdep.cpu_idle_hlt and SMP perf?
Message-ID:  <17379.56708.421007.613310@grasshopper.cs.duke.edu>

next in thread | raw e-mail | index | archive | help

Why dooes machdep.cpu_idle_hlt=1 drop my 10GbE network rx
performance by a considerable amount (7.5Gbs -> 5.5Gbs)?

I've (blindly) tried leaving machdep.cpu_idle_hlt=1 enabled
and playing with the vast array of kern.sched.ipiwakeup.* sysctls,
but receive performance remains limited to ~5.5Gb/sec or less.

This is an 'AMD Athlon(tm) 64 X2 Dual Core Processor 3800+' running
FreeBSD-current as of about one week ago.  The interrupt load is 
about 22,000 device interrupts/sec (ithreaded).  Interestingly,
the more I decrease the interrupt load by increasing the interrupt
coalescing timer, the worse the machdep.cpu_idle_hlt=1 case does.

Is this just a case of the wakeup IPI taking a long time or blocking
on some lock?

Drew

PS: Here is what I mean:

rome% ssh venice-my netperf224 -Hrome-my -tTCP_SENDFILE -F /boot/vmlinuz-2.6.9-11.EL -- -S 131072
TCP SENDFILE TEST to rome-my
Recv   Send    Send                          
Socket Socket  Message  Elapsed              
Size   Size    Size     Time     Throughput  
bytes  bytes   bytes    secs.    10^6bits/sec  

131072  65536  65536    10.00    5460.73   

rome% sudo sysctl machdep.cpu_idle_hlt=0
machdep.cpu_idle_hlt: 1 -> 0

rome% ssh venice-my netperf224 -Hrome-my -tTCP_SENDFILE -F /boot/vmlinuz-2.6.9-11.EL -- -S 131072
TCP SENDFILE TEST to rome-my
Recv   Send    Send                          
Socket Socket  Message  Elapsed              
Size   Size    Size     Time     Throughput  
bytes  bytes   bytes    secs.    10^6bits/sec  

131072  65536  65536    10.00    7842.41   



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?17379.56708.421007.613310>