Date: Thu, 2 Apr 2009 18:19:30 +0800 From: =?GB2312?B?tN7R0mNjdWl5eWFuQHNpbmEuY29t?= <ccuiyyan@gmail.com> To: freebsd-current@freebsd.org Subject: dup() scales badly on multicore platform Message-ID: <4451ccf20904020319r6b66f390vebbb678b9e8eb2ab@mail.gmail.com>
index | next in thread | raw e-mail
[-- Attachment #1 --]
Dear all;
i benchchmark the dup() system call on 32 cores machine in
FreeBSD-current8.0.
The results are bad. The phenomenon is easy to come out. Each
process(not thread) on the core
dup() its private file, and close() in a tight loop. The time
completing the parallel workload
is considered as performance. At first, i think there are some
locks. However, lock profiling
in FreeBSD is strange and interesting. I attach the graph and lock
profiling. Any ideas?
[-- Attachment #2 --]
mutex:process lock)
149 408 12180876 213508 146766 82 1 0 1322 /usr/src/sys/kern/kern_clock.c:286 (spin mutex:sched lock 1)
2790 2240 453750 217852 710 639 306 0 334 /usr/src/sys/kern/kern_mutex.c:137 (sleep mutex:USB device mutex)
31220 20331 466506511 223166 1600017 291 0 0 353 /usr/src/sys/kern/kern_descrip.c:710 (sleep mutex:process lock)
153 601 12195293 258655 146277 83 1 0 2039 /usr/src/sys/kern/kern_clock.c:286 (spin mutex:sched lock 2)
151 395 12208207 268080 146745 83 1 0 2165 /usr/src/sys/kern/kern_clock.c:286 (spin mutex:sched lock 3)
7974 52578 298977 276284 1212 246 227 0 176 /usr/src/sys/vm/vm_glue.c:689 (sleep mutex:process lock)
2332 13892 147200 302482 710 207 426 0 136 /usr/src/sys/dev/usb2/core/usb2_request.c:474 (sleep mutex:Giant)
8737 15271 4299801 338888 25016 171 13 0 769 /usr/src/sys/vm/uma_core.c:2524 (sleep mutex:g_bio)
15498 13395 3934241 343776 25016 157 13 0 779 /usr/src/sys/vm/uma_core.c:1990 (sleep mutex:g_bio)
162 356 419733 356862 4579 91 77 0 3547 /usr/src/sys/kern/sched_ule.c:2074 (spin mutex:sched lock 3)
1165 12708 470365 387472 713 659 543 0 76 /usr/src/sys/kern/subr_turnstile.c:547 (spin mutex:turnstile chain)
254 916 844616 388616 8279 102 46 0 2864 /usr/src/sys/kern/kern_intr.c:800 (spin mutex:sched lock 1)
6956 18191 1511441 402382 3169 476 126 0 228 /usr/src/sys/vm/vm_fault.c:1007 (sleep mutex:vm object)
136 130565 28795 433914 332 86 1306 0 302 /usr/src/sys/kern/sched_ule.c:2569 (spin mutex:sched lock 1)
176 446 494854 464029 5375 92 86 0 4487 /usr/src/sys/kern/sched_ule.c:2074 (spin mutex:sched lock 2)
38417 7562 13164154 469326 5766 2283 81 0 689 /usr/src/sys/kern/kern_mutex.c:137 (sleep mutex:Giant)
30603 9639 12304039 538138 12508 983 43 0 419 /usr/src/sys/dev/ata/ata-queue.c:177 (sleep mutex:ATA queue lock)
164 419 581799 553747 6732 86 82 0 5672 /usr/src/sys/kern/sched_ule.c:2074 (spin mutex:sched lock 1)
195 9941 2829970 616121 32150 88 19 0 3494 /usr/src/sys/kern/kern_timeout.c:436 (spin mutex:callout)
182 4983 2243023 672841 25374 88 26 0 3584 /usr/src/sys/kern/kern_timeout.c:325 (spin mutex:callout)
5961 22422 490335 780349 3116 157 250 0 945 /usr/src/sys/kern/kern_mutex.c:137 (sleep mutex:sleep mtxpool)
6763 89799 1123998 804901 3482 322 231 0 122 /usr/src/sys/vm/vm_object.c:447 (sleep mutex:vm object)
17068 15023 24074540 916282 98813 243 9 0 1414 /usr/src/sys/geom/geom_io.c:68 (sleep mutex:bio queue)
8014 13019 792214 1146370 1119 707 1024 0 698 /usr/src/sys/kern/kern_mutex.c:137 (sleep mutex:ohci0)
5210 8052 753070 1191131 1066 706 1117 0 712 /usr/src/sys/kern/kern_mutex.c:137 (sleep mutex:ehci0)
8733 10715 3109705 1229676 1333 2332 922 0 342 /usr/src/sys/dev/usb2/core/usb2_transfer.c:1809 (sleep mutex:USB device mutex)
212 1750 5995122 1452339 69777 85 20 0 8658 /usr/src/sys/kern/kern_timeout.c:272 (spin mutex:callout)
4458 17933 26800701 1520828 93790 285 16 0 3149 /usr/src/sys/kern/subr_sleepqueue.c:236 (spin mutex:sleepq chain)
250 608 569869085 1706686 4774453 119 0 0 12389 /usr/src/sys/kern/kern_timeout.c:246 (spin mutex:callout)
4463 16718 3315242 1725846 5334 621 323 0 771 /usr/src/sys/kern/kern_intr.c:1133 (sleep mutex:Giant)
1809 219449 129038 2286857 211 611 10838 0 209 /usr/src/sys/vm/vm_fault.c:375 (sleep mutex:vm object)
924 164275 36963 2554204 200 184 12771 0 198 /usr/src/sys/kern/kern_mutex.c:137 (sleep mutex:vm object)
9545 367923 1187075 2814977 2155 550 1306 0 238 /usr/src/sys/vm/vm_fault.c:297 (sleep mutex:vm object)
1678 18926 2260649 3208378 7872 287 407 0 725 /usr/src/sys/kern/kern_conf.c:422 (sleep mutex:Giant)
1683 101889 2267373 3829747 7937 285 482 0 784 /usr/src/sys/kern/kern_conf.c:454 (sleep mutex:Giant)
help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4451ccf20904020319r6b66f390vebbb678b9e8eb2ab>