From owner-freebsd-current@FreeBSD.ORG Thu Apr 2 10:19:31 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1E875106566C for ; Thu, 2 Apr 2009 10:19:31 +0000 (UTC) (envelope-from ccuiyyan@gmail.com) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.178]) by mx1.freebsd.org (Postfix) with ESMTP id E28DC8FC0C for ; Thu, 2 Apr 2009 10:19:30 +0000 (UTC) (envelope-from ccuiyyan@gmail.com) Received: by wa-out-1112.google.com with SMTP id m38so312285waf.27 for ; Thu, 02 Apr 2009 03:19:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=W4HQd2Dwlq6rp8pHQ27UqmKhN25fF636/5oQIWusMrM=; b=M5cqKjz2+7YNeOeTpO+/qhnrbJ+zE77Vg/2LRlIXkAkXoLIIUfOmIyTzhkS43s4dzN aEWP6c38rqrZwl4y7kqGmslwQ4/8edemywrat4WpbxA5Jq4Vn8pYvPJ+dIvGVsw7Vtr5 2eOIuNzXuW8AVZDgsVEnqbmc8Md5vnn8rEP9M= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=RwreyuCSj2Tn4kNkYz26RNLWxBgA2e/4oAv++kuM8TCoYYA9pyTbWKhvTK0qMtaMuQ XmwZl2fA8DfCZMezJbY1XEDyp6QUJWO6A+NTbBpaUGgec9dM4po5NAhrcyVUQqzj3n52 fDWhhgUr5MGWoaXUgBwwJr55rk8LHVwtN8lhA= MIME-Version: 1.0 Received: by 10.114.133.1 with SMTP id g1mr5813735wad.162.1238667570692; Thu, 02 Apr 2009 03:19:30 -0700 (PDT) Date: Thu, 2 Apr 2009 18:19:30 +0800 Message-ID: <4451ccf20904020319r6b66f390vebbb678b9e8eb2ab@mail.gmail.com> From: =?GB2312?B?tN7R0mNjdWl5eWFuQHNpbmEuY29t?= To: freebsd-current@freebsd.org Content-Type: multipart/mixed; boundary=0016e64dd4ce8bce0604668fc488 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: dup() scales badly on multicore platform X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2009 10:19:31 -0000 --0016e64dd4ce8bce0604668fc488 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Dear all; i benchchmark the dup() system call on 32 cores machine in FreeBSD-current8.0. The results are bad. The phenomenon is easy to come out. Each process(not thread) on the core dup() its private file, and close() in a tight loop. The time completing the parallel workload is considered as performance. At first, i think there are some locks. However, lock profiling in FreeBSD is strange and interesting. I attach the graph and lock profiling. Any ideas? --0016e64dd4ce8bce0604668fc488 Content-Type: application/octet-stream; name=TCCC Content-Disposition: attachment; filename=TCCC Content-Transfer-Encoding: base64 X-Attachment-Id: f_ft1anyhd1 IG11dGV4OnByb2Nlc3MgbG9jaykKICAgICAxNDkgICAgICAgNDA4ICAgIDEyMTgwODc2ICAgICAg MjEzNTA4ICAgICAgMTQ2NzY2ICAgICA4MiAgICAgIDEgIDAgICAxMzIyIC91c3Ivc3JjL3N5cy9r ZXJuL2tlcm5fY2xvY2suYzoyODYgKHNwaW4gbXV0ZXg6c2NoZWQgbG9jayAxKQogICAgMjc5MCAg ICAgIDIyNDAgICAgICA0NTM3NTAgICAgICAyMTc4NTIgICAgICAgICA3MTAgICAgNjM5ICAgIDMw NiAgMCAgICAzMzQgL3Vzci9zcmMvc3lzL2tlcm4va2Vybl9tdXRleC5jOjEzNyAoc2xlZXAgbXV0 ZXg6VVNCIGRldmljZSBtdXRleCkKICAgMzEyMjAgICAgIDIwMzMxICAgNDY2NTA2NTExICAgICAg MjIzMTY2ICAgICAxNjAwMDE3ICAgIDI5MSAgICAgIDAgIDAgICAgMzUzIC91c3Ivc3JjL3N5cy9r ZXJuL2tlcm5fZGVzY3JpcC5jOjcxMCAoc2xlZXAgbXV0ZXg6cHJvY2VzcyBsb2NrKQogICAgIDE1 MyAgICAgICA2MDEgICAgMTIxOTUyOTMgICAgICAyNTg2NTUgICAgICAxNDYyNzcgICAgIDgzICAg ICAgMSAgMCAgIDIwMzkgL3Vzci9zcmMvc3lzL2tlcm4va2Vybl9jbG9jay5jOjI4NiAoc3BpbiBt dXRleDpzY2hlZCBsb2NrIDIpCiAgICAgMTUxICAgICAgIDM5NSAgICAxMjIwODIwNyAgICAgIDI2 ODA4MCAgICAgIDE0Njc0NSAgICAgODMgICAgICAxICAwICAgMjE2NSAvdXNyL3NyYy9zeXMva2Vy bi9rZXJuX2Nsb2NrLmM6Mjg2IChzcGluIG11dGV4OnNjaGVkIGxvY2sgMykKICAgIDc5NzQgICAg IDUyNTc4ICAgICAgMjk4OTc3ICAgICAgMjc2Mjg0ICAgICAgICAxMjEyICAgIDI0NiAgICAyMjcg IDAgICAgMTc2IC91c3Ivc3JjL3N5cy92bS92bV9nbHVlLmM6Njg5IChzbGVlcCBtdXRleDpwcm9j ZXNzIGxvY2spCiAgICAyMzMyICAgICAxMzg5MiAgICAgIDE0NzIwMCAgICAgIDMwMjQ4MiAgICAg ICAgIDcxMCAgICAyMDcgICAgNDI2ICAwICAgIDEzNiAvdXNyL3NyYy9zeXMvZGV2L3VzYjIvY29y ZS91c2IyX3JlcXVlc3QuYzo0NzQgKHNsZWVwIG11dGV4OkdpYW50KQogICAgODczNyAgICAgMTUy NzEgICAgIDQyOTk4MDEgICAgICAzMzg4ODggICAgICAgMjUwMTYgICAgMTcxICAgICAxMyAgMCAg ICA3NjkgL3Vzci9zcmMvc3lzL3ZtL3VtYV9jb3JlLmM6MjUyNCAoc2xlZXAgbXV0ZXg6Z19iaW8p CiAgIDE1NDk4ICAgICAxMzM5NSAgICAgMzkzNDI0MSAgICAgIDM0Mzc3NiAgICAgICAyNTAxNiAg ICAxNTcgICAgIDEzICAwICAgIDc3OSAvdXNyL3NyYy9zeXMvdm0vdW1hX2NvcmUuYzoxOTkwIChz bGVlcCBtdXRleDpnX2JpbykKICAgICAxNjIgICAgICAgMzU2ICAgICAgNDE5NzMzICAgICAgMzU2 ODYyICAgICAgICA0NTc5ICAgICA5MSAgICAgNzcgIDAgICAzNTQ3IC91c3Ivc3JjL3N5cy9rZXJu L3NjaGVkX3VsZS5jOjIwNzQgKHNwaW4gbXV0ZXg6c2NoZWQgbG9jayAzKQogICAgMTE2NSAgICAg MTI3MDggICAgICA0NzAzNjUgICAgICAzODc0NzIgICAgICAgICA3MTMgICAgNjU5ICAgIDU0MyAg MCAgICAgNzYgL3Vzci9zcmMvc3lzL2tlcm4vc3Vicl90dXJuc3RpbGUuYzo1NDcgKHNwaW4gbXV0 ZXg6dHVybnN0aWxlIGNoYWluKQogICAgIDI1NCAgICAgICA5MTYgICAgICA4NDQ2MTYgICAgICAz ODg2MTYgICAgICAgIDgyNzkgICAgMTAyICAgICA0NiAgMCAgIDI4NjQgL3Vzci9zcmMvc3lzL2tl cm4va2Vybl9pbnRyLmM6ODAwIChzcGluIG11dGV4OnNjaGVkIGxvY2sgMSkKICAgIDY5NTYgICAg IDE4MTkxICAgICAxNTExNDQxICAgICAgNDAyMzgyICAgICAgICAzMTY5ICAgIDQ3NiAgICAxMjYg IDAgICAgMjI4IC91c3Ivc3JjL3N5cy92bS92bV9mYXVsdC5jOjEwMDcgKHNsZWVwIG11dGV4OnZt IG9iamVjdCkKICAgICAxMzYgICAgMTMwNTY1ICAgICAgIDI4Nzk1ICAgICAgNDMzOTE0ICAgICAg ICAgMzMyICAgICA4NiAgIDEzMDYgIDAgICAgMzAyIC91c3Ivc3JjL3N5cy9rZXJuL3NjaGVkX3Vs ZS5jOjI1NjkgKHNwaW4gbXV0ZXg6c2NoZWQgbG9jayAxKQogICAgIDE3NiAgICAgICA0NDYgICAg ICA0OTQ4NTQgICAgICA0NjQwMjkgICAgICAgIDUzNzUgICAgIDkyICAgICA4NiAgMCAgIDQ0ODcg L3Vzci9zcmMvc3lzL2tlcm4vc2NoZWRfdWxlLmM6MjA3NCAoc3BpbiBtdXRleDpzY2hlZCBsb2Nr IDIpCiAgIDM4NDE3ICAgICAgNzU2MiAgICAxMzE2NDE1NCAgICAgIDQ2OTMyNiAgICAgICAgNTc2 NiAgIDIyODMgICAgIDgxICAwICAgIDY4OSAvdXNyL3NyYy9zeXMva2Vybi9rZXJuX211dGV4LmM6 MTM3IChzbGVlcCBtdXRleDpHaWFudCkKICAgMzA2MDMgICAgICA5NjM5ICAgIDEyMzA0MDM5ICAg ICAgNTM4MTM4ICAgICAgIDEyNTA4ICAgIDk4MyAgICAgNDMgIDAgICAgNDE5IC91c3Ivc3JjL3N5 cy9kZXYvYXRhL2F0YS1xdWV1ZS5jOjE3NyAoc2xlZXAgbXV0ZXg6QVRBIHF1ZXVlIGxvY2spCiAg ICAgMTY0ICAgICAgIDQxOSAgICAgIDU4MTc5OSAgICAgIDU1Mzc0NyAgICAgICAgNjczMiAgICAg ODYgICAgIDgyICAwICAgNTY3MiAvdXNyL3NyYy9zeXMva2Vybi9zY2hlZF91bGUuYzoyMDc0IChz cGluIG11dGV4OnNjaGVkIGxvY2sgMSkKICAgICAxOTUgICAgICA5OTQxICAgICAyODI5OTcwICAg ICAgNjE2MTIxICAgICAgIDMyMTUwICAgICA4OCAgICAgMTkgIDAgICAzNDk0IC91c3Ivc3JjL3N5 cy9rZXJuL2tlcm5fdGltZW91dC5jOjQzNiAoc3BpbiBtdXRleDpjYWxsb3V0KQogICAgIDE4MiAg ICAgIDQ5ODMgICAgIDIyNDMwMjMgICAgICA2NzI4NDEgICAgICAgMjUzNzQgICAgIDg4ICAgICAy NiAgMCAgIDM1ODQgL3Vzci9zcmMvc3lzL2tlcm4va2Vybl90aW1lb3V0LmM6MzI1IChzcGluIG11 dGV4OmNhbGxvdXQpCiAgICA1OTYxICAgICAyMjQyMiAgICAgIDQ5MDMzNSAgICAgIDc4MDM0OSAg ICAgICAgMzExNiAgICAxNTcgICAgMjUwICAwICAgIDk0NSAvdXNyL3NyYy9zeXMva2Vybi9rZXJu X211dGV4LmM6MTM3IChzbGVlcCBtdXRleDpzbGVlcCBtdHhwb29sKQogICAgNjc2MyAgICAgODk3 OTkgICAgIDExMjM5OTggICAgICA4MDQ5MDEgICAgICAgIDM0ODIgICAgMzIyICAgIDIzMSAgMCAg ICAxMjIgL3Vzci9zcmMvc3lzL3ZtL3ZtX29iamVjdC5jOjQ0NyAoc2xlZXAgbXV0ZXg6dm0gb2Jq ZWN0KQogICAxNzA2OCAgICAgMTUwMjMgICAgMjQwNzQ1NDAgICAgICA5MTYyODIgICAgICAgOTg4 MTMgICAgMjQzICAgICAgOSAgMCAgIDE0MTQgL3Vzci9zcmMvc3lzL2dlb20vZ2VvbV9pby5jOjY4 IChzbGVlcCBtdXRleDpiaW8gcXVldWUpCiAgICA4MDE0ICAgICAxMzAxOSAgICAgIDc5MjIxNCAg ICAgMTE0NjM3MCAgICAgICAgMTExOSAgICA3MDcgICAxMDI0ICAwICAgIDY5OCAvdXNyL3NyYy9z eXMva2Vybi9rZXJuX211dGV4LmM6MTM3IChzbGVlcCBtdXRleDpvaGNpMCkKICAgIDUyMTAgICAg ICA4MDUyICAgICAgNzUzMDcwICAgICAxMTkxMTMxICAgICAgICAxMDY2ICAgIDcwNiAgIDExMTcg IDAgICAgNzEyIC91c3Ivc3JjL3N5cy9rZXJuL2tlcm5fbXV0ZXguYzoxMzcgKHNsZWVwIG11dGV4 OmVoY2kwKQogICAgODczMyAgICAgMTA3MTUgICAgIDMxMDk3MDUgICAgIDEyMjk2NzYgICAgICAg IDEzMzMgICAyMzMyICAgIDkyMiAgMCAgICAzNDIgL3Vzci9zcmMvc3lzL2Rldi91c2IyL2NvcmUv dXNiMl90cmFuc2Zlci5jOjE4MDkgKHNsZWVwIG11dGV4OlVTQiBkZXZpY2UgbXV0ZXgpCiAgICAg MjEyICAgICAgMTc1MCAgICAgNTk5NTEyMiAgICAgMTQ1MjMzOSAgICAgICA2OTc3NyAgICAgODUg ICAgIDIwICAwICAgODY1OCAvdXNyL3NyYy9zeXMva2Vybi9rZXJuX3RpbWVvdXQuYzoyNzIgKHNw aW4gbXV0ZXg6Y2FsbG91dCkKICAgIDQ0NTggICAgIDE3OTMzICAgIDI2ODAwNzAxICAgICAxNTIw ODI4ICAgICAgIDkzNzkwICAgIDI4NSAgICAgMTYgIDAgICAzMTQ5IC91c3Ivc3JjL3N5cy9rZXJu L3N1YnJfc2xlZXBxdWV1ZS5jOjIzNiAoc3BpbiBtdXRleDpzbGVlcHEgY2hhaW4pCiAgICAgMjUw ICAgICAgIDYwOCAgIDU2OTg2OTA4NSAgICAgMTcwNjY4NiAgICAgNDc3NDQ1MyAgICAxMTkgICAg ICAwICAwICAxMjM4OSAvdXNyL3NyYy9zeXMva2Vybi9rZXJuX3RpbWVvdXQuYzoyNDYgKHNwaW4g bXV0ZXg6Y2FsbG91dCkKICAgIDQ0NjMgICAgIDE2NzE4ICAgICAzMzE1MjQyICAgICAxNzI1ODQ2 ICAgICAgICA1MzM0ICAgIDYyMSAgICAzMjMgIDAgICAgNzcxIC91c3Ivc3JjL3N5cy9rZXJuL2tl cm5faW50ci5jOjExMzMgKHNsZWVwIG11dGV4OkdpYW50KQogICAgMTgwOSAgICAyMTk0NDkgICAg ICAxMjkwMzggICAgIDIyODY4NTcgICAgICAgICAyMTEgICAgNjExICAxMDgzOCAgMCAgICAyMDkg L3Vzci9zcmMvc3lzL3ZtL3ZtX2ZhdWx0LmM6Mzc1IChzbGVlcCBtdXRleDp2bSBvYmplY3QpCiAg ICAgOTI0ICAgIDE2NDI3NSAgICAgICAzNjk2MyAgICAgMjU1NDIwNCAgICAgICAgIDIwMCAgICAx ODQgIDEyNzcxICAwICAgIDE5OCAvdXNyL3NyYy9zeXMva2Vybi9rZXJuX211dGV4LmM6MTM3IChz bGVlcCBtdXRleDp2bSBvYmplY3QpCiAgICA5NTQ1ICAgIDM2NzkyMyAgICAgMTE4NzA3NSAgICAg MjgxNDk3NyAgICAgICAgMjE1NSAgICA1NTAgICAxMzA2ICAwICAgIDIzOCAvdXNyL3NyYy9zeXMv dm0vdm1fZmF1bHQuYzoyOTcgKHNsZWVwIG11dGV4OnZtIG9iamVjdCkKICAgIDE2NzggICAgIDE4 OTI2ICAgICAyMjYwNjQ5ICAgICAzMjA4Mzc4ICAgICAgICA3ODcyICAgIDI4NyAgICA0MDcgIDAg ICAgNzI1IC91c3Ivc3JjL3N5cy9rZXJuL2tlcm5fY29uZi5jOjQyMiAoc2xlZXAgbXV0ZXg6R2lh bnQpCiAgICAxNjgzICAgIDEwMTg4OSAgICAgMjI2NzM3MyAgICAgMzgyOTc0NyAgICAgICAgNzkz NyAgICAyODUgICAgNDgyICAwICAgIDc4NCAvdXNyL3NyYy9zeXMva2Vybi9rZXJuX2NvbmYuYzo0 NTQgKHNsZWVwIG11dGV4OkdpYW50KQo= --0016e64dd4ce8bce0604668fc488--