From owner-freebsd-fs@freebsd.org Sat Jan 9 02:41:25 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4A94FA68BB2 for ; Sat, 9 Jan 2016 02:41:25 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id DA83815D7 for ; Sat, 9 Jan 2016 02:41:24 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:sBBklxKSxjKlN7v50dmcpTZWNBhigK39O0sv0rFitYgULPnxwZ3uMQTl6Ol3ixeRBMOAu6wC07KempujcFJDyK7JiGoFfp1IWk1NouQttCtkPvS4D1bmJuXhdS0wEZcKflZk+3amLRodQ56mNBXsq3G/pQQfBg/4fVIsYL+lRMiK14ye7KObxd76W01wnj2zYLd/fl2djD76kY0ou7ZkMbs70RDTo3FFKKx8zGJsIk+PzV6nvp/jtLYqySlbuuog+shcSu26Ov1gFf0LRAghZkI46sOjmRDZRhrHsnkQW38dgzJSDgTF5Q28VZD05HjUrO14jRObNs6+aLk/WjCv6u8/UhrhgyQDOjsR7WbYl8F0lKIdqxv39E83+JLdfIzAbKk2RajaZ95PADMZBss= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CuBABYcpBW/61jaINdhAxtBohTtT0YCoUjSgKBVxEBAQEBAQEBAYEJgi2CCAEBBAEBASAEJyALDAQCAQgOGxkCAgIlAQkmAgQIBwQBHASIDg6xM5AyAQEBAQEBAQEBAQEBAQEBAQEBEgUEgi+EJ4R/hDcBARsZgwaBSQWOMIhdgnSCT4UphEiHbYUyjkwCOCyCERyBeyA0B4QYOoEIAQEB X-IronPort-AV: E=Sophos;i="5.20,541,1444708800"; d="scan'208";a="261928249" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 08 Jan 2016 21:40:16 -0500 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 1CCA315F55D; Fri, 8 Jan 2016 21:40:16 -0500 (EST) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id WWiq5ZaflpHb; Fri, 8 Jan 2016 21:40:15 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 53C1715F565; Fri, 8 Jan 2016 21:40:15 -0500 (EST) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 67obn-TwXJgx; Fri, 8 Jan 2016 21:40:15 -0500 (EST) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 2E87715F55D; Fri, 8 Jan 2016 21:40:15 -0500 (EST) Date: Fri, 8 Jan 2016 21:40:15 -0500 (EST) From: Rick Macklem To: Jeff Darcy Cc: Raghavendra G , freebsd-fs , Hubbard Jordan , Xavier Hernandez , Gluster Devel Message-ID: <2013962695.154259810.1452307215168.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <1924941590.6473225.1452248249994.JavaMail.zimbra@redhat.com> References: <571237035.145690509.1451437960464.JavaMail.zimbra@uoguelph.ca> <20151230103152.GS13942@ndevos-x240.usersys.redhat.com> <2D8C2729-D556-479B-B4E2-66E1BB222F41@ixsystems.com> <1083933309.146084334.1451517977647.JavaMail.zimbra@uoguelph.ca> <568F6D07.6070500@datalab.es> <1924941590.6473225.1452248249994.JavaMail.zimbra@redhat.com> Subject: Re: [Gluster-devel] FreeBSD port of GlusterFS racks up a lot of CPU usage MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_154259808_2047258858.1452307215166" X-Originating-IP: [172.17.95.12] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF43 (Win)/8.0.9_GA_6191) Thread-Topic: FreeBSD port of GlusterFS racks up a lot of CPU usage Thread-Index: nzgvBLPPgcBXuRsf6GlZU17foVfxCMhA9xbs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jan 2016 02:41:25 -0000 ------=_Part_154259808_2047258858.1452307215166 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Oops, I realized the last patch did a write(2) while holding a pthread_mutex. I've never used pthread_mutexes, but I suspect this isn't allowed. The attached updated patch delays the write() until after the pthread_mutex_unlock(). Sorry about the confusion, rick ----- Original Message ----- > > > I don't know anything about gluster's poll implementation so I may > > > be totally wrong, but would it be possible to use an eventfd (or a > > > pipe if eventfd is not supported) to signal the need to add more > > > file descriptors to the poll call ? > > > > > > > > > The poll call should listen on this new fd. When we need to change > > > the fd list, we should simply write to the eventfd or pipe from > > > another thread. This will cause the poll call to return and we will > > > be able to change the fd list without having a short timeout nor > > > having to decide on any trade-off. > > > > > > Thats a nice idea. Based on my understanding of why timeouts are being > > used, this approach can work. > > The own-thread code which preceded the current poll implementation did > something similar, using a pipe fd to be woken up for new *outgoing* > messages. That code still exists, and might provide some insight into > how to do this for the current poll code. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > ------=_Part_154259808_2047258858.1452307215166 Content-Type: text/x-patch; name=gluster-poll.patch Content-Disposition: attachment; filename=gluster-poll.patch Content-Transfer-Encoding: base64 LS0tIGdsdXN0ZXJmcy0zLjcuNi9saWJnbHVzdGVyZnMvc3JjL2V2ZW50LXBvbGwuYy5zYXYJMjAx Ni0wMS0wNiAxNTo1ODowMy41MjIyODYwMDAgLTA1MDAKKysrIGdsdXN0ZXJmcy0zLjcuNi9saWJn bHVzdGVyZnMvc3JjL2V2ZW50LXBvbGwuYwkyMDE2LTAxLTA4IDE4OjE0OjU3LjY1ODY1MjAwMCAt MDUwMApAQCAtMTgwLDYgKzE4MCwxNSBAQCBldmVudF9wb29sX25ld19wb2xsIChpbnQgY291bnQs IGludCBldmVuCiAgICAgICAgIHJldHVybiBldmVudF9wb29sOwogfQogCitzdGF0aWMgdm9pZAor ZXZlbnRfcG9vbF9jaGFuZ2VkIChzdHJ1Y3QgZXZlbnRfcG9vbCAqZXZlbnRfcG9vbCkKK3sKKwor ICAgICAgICAvKiBXcml0ZSBhIGJ5dGUgaW50byB0aGUgYnJlYWtlciBwaXBlIHRvIHdha2UgdXAg cG9sbCgpLiAqLworICAgICAgICBpZiAoZXZlbnRfcG9vbC0+YnJlYWtlclsxXSA+PSAwKQorICAg ICAgICAgICAgICAgIHdyaXRlKGV2ZW50X3Bvb2wtPmJyZWFrZXJbMV0sICJYIiwgMSk7Cit9CisK IAogc3RhdGljIGludAogZXZlbnRfcmVnaXN0ZXJfcG9sbCAoc3RydWN0IGV2ZW50X3Bvb2wgKmV2 ZW50X3Bvb2wsIGludCBmZCwKQEAgLTE4Nyw2ICsxOTYsNyBAQCBldmVudF9yZWdpc3Rlcl9wb2xs IChzdHJ1Y3QgZXZlbnRfcG9vbCAqCiAgICAgICAgICAgICAgICAgICAgICB2b2lkICpkYXRhLCBp bnQgcG9sbF9pbiwgaW50IHBvbGxfb3V0KQogewogICAgICAgICBpbnQgaWR4ID0gLTE7CisgICAg ICAgIGludCBjaGFuZ2VkID0gMDsKIAogICAgICAgICBHRl9WQUxJREFURV9PUl9HT1RPICgiZXZl bnQiLCBldmVudF9wb29sLCBvdXQpOwogCkBAIC0yNDUsMTAgKzI1NSwxMyBAQCBldmVudF9yZWdp c3Rlcl9wb2xsIChzdHJ1Y3QgZXZlbnRfcG9vbCAqCiAgICAgICAgICAgICAgICAgfQogCiAgICAg ICAgICAgICAgICAgZXZlbnRfcG9vbC0+Y2hhbmdlZCA9IDE7CisgICAgICAgICAgICAgICAgY2hh bmdlZCA9IDE7CiAKICAgICAgICAgfQogdW5sb2NrOgogICAgICAgICBwdGhyZWFkX211dGV4X3Vu bG9jayAoJmV2ZW50X3Bvb2wtPm11dGV4KTsKKyAgICAgICAgaWYgKGNoYW5nZWQgIT0gMCkKKyAg ICAgICAgICAgICAgICBldmVudF9wb29sX2NoYW5nZWQoZXZlbnRfcG9vbCk7CiAKIG91dDoKICAg ICAgICAgcmV0dXJuIGlkeDsKQEAgLTI1OSw2ICsyNzIsNyBAQCBzdGF0aWMgaW50CiBldmVudF91 bnJlZ2lzdGVyX3BvbGwgKHN0cnVjdCBldmVudF9wb29sICpldmVudF9wb29sLCBpbnQgZmQsIGlu dCBpZHhfaGludCkKIHsKICAgICAgICAgaW50IGlkeCA9IC0xOworICAgICAgICBpbnQgY2hhbmdl ZCA9IDA7CiAKICAgICAgICAgR0ZfVkFMSURBVEVfT1JfR09UTyAoImV2ZW50IiwgZXZlbnRfcG9v bCwgb3V0KTsKIApAQCAtMjc2LDkgKzI5MCwxMiBAQCBldmVudF91bnJlZ2lzdGVyX3BvbGwgKHN0 cnVjdCBldmVudF9wb29sCiAKICAgICAgICAgICAgICAgICBldmVudF9wb29sLT5yZWdbaWR4XSA9 ICBldmVudF9wb29sLT5yZWdbLS1ldmVudF9wb29sLT51c2VkXTsKICAgICAgICAgICAgICAgICBl dmVudF9wb29sLT5jaGFuZ2VkID0gMTsKKyAgICAgICAgICAgICAgICBjaGFuZ2VkID0gMTsKICAg ICAgICAgfQogdW5sb2NrOgogICAgICAgICBwdGhyZWFkX211dGV4X3VubG9jayAoJmV2ZW50X3Bv b2wtPm11dGV4KTsKKyAgICAgICAgaWYgKGNoYW5nZWQgIT0gMCkKKyAgICAgICAgICAgICAgICBl dmVudF9wb29sX2NoYW5nZWQoZXZlbnRfcG9vbCk7CiAKIG91dDoKICAgICAgICAgcmV0dXJuIGlk eDsKQEAgLTMwNCw2ICszMjEsNyBAQCBldmVudF9zZWxlY3Rfb25fcG9sbCAoc3RydWN0IGV2ZW50 X3Bvb2wgCiAgICAgICAgICAgICAgICAgICAgICAgaW50IHBvbGxfaW4sIGludCBwb2xsX291dCkK IHsKICAgICAgICAgaW50IGlkeCA9IC0xOworICAgICAgICBpbnQgY2hhbmdlZCA9IDA7CiAKICAg ICAgICAgR0ZfVkFMSURBVEVfT1JfR09UTyAoImV2ZW50IiwgZXZlbnRfcG9vbCwgb3V0KTsKIApA QCAtMzQ5LDExICszNjcsMTUgQEAgZXZlbnRfc2VsZWN0X29uX3BvbGwgKHN0cnVjdCBldmVudF9w b29sIAogICAgICAgICAgICAgICAgICAgICAgICAgYnJlYWs7CiAgICAgICAgICAgICAgICAgfQog Ci0gICAgICAgICAgICAgICAgaWYgKHBvbGxfaW4gKyBwb2xsX291dCA+IC0yKQorICAgICAgICAg ICAgICAgIGlmIChwb2xsX2luICsgcG9sbF9vdXQgPiAtMikgewogICAgICAgICAgICAgICAgICAg ICAgICAgZXZlbnRfcG9vbC0+Y2hhbmdlZCA9IDE7CisgICAgICAgICAgICAgICAgICAgICAgICBj aGFuZ2VkID0gMTsKKyAgICAgICAgICAgICAgICB9CiAgICAgICAgIH0KIHVubG9jazoKICAgICAg ICAgcHRocmVhZF9tdXRleF91bmxvY2sgKCZldmVudF9wb29sLT5tdXRleCk7CisgICAgICAgIGlm IChjaGFuZ2VkICE9IDApCisgICAgICAgICAgICAgICAgZXZlbnRfcG9vbF9jaGFuZ2VkKGV2ZW50 X3Bvb2wpOwogCiBvdXQ6CiAgICAgICAgIHJldHVybiBpZHg7CkBAIC00NDgsNiArNDcwLDcgQEAg ZXZlbnRfZGlzcGF0Y2hfcG9sbCAoc3RydWN0IGV2ZW50X3Bvb2wgKgogICAgICAgICBpbnQgICAg ICAgICAgICAgIHNpemUgPSAwOwogICAgICAgICBpbnQgICAgICAgICAgICAgIGkgPSAwOwogICAg ICAgICBpbnQgICAgICAgICAgICAgIHJldCA9IC0xOworICAgICAgICBjaGFyICAgICAgICAgICAg IHg7CiAKICAgICAgICAgR0ZfVkFMSURBVEVfT1JfR09UTyAoImV2ZW50IiwgZXZlbnRfcG9vbCwg b3V0KTsKIApAQCAtNDcyLDcgKzQ5NSw3IEBAIGV2ZW50X2Rpc3BhdGNoX3BvbGwgKHN0cnVjdCBl dmVudF9wb29sICoKICAgICAgICAgICAgICAgICBzaXplID0gZXZlbnRfZGlzcGF0Y2hfcG9sbF9y ZXNpemUgKGV2ZW50X3Bvb2wsIHVmZHMsIHNpemUpOwogICAgICAgICAgICAgICAgIHVmZHMgPSBl dmVudF9wb29sLT5ldmNhY2hlOwogCi0gICAgICAgICAgICAgICAgcmV0ID0gcG9sbCAodWZkcywg c2l6ZSwgMSk7CisgICAgICAgICAgICAgICAgcmV0ID0gcG9sbCAodWZkcywgc2l6ZSwgLTEpOwog CiAgICAgICAgICAgICAgICAgaWYgKHJldCA9PSAwKQogICAgICAgICAgICAgICAgICAgICAgICAg LyogdGltZW91dCAqLwpAQCAtNDgyLDcgKzUwNSwxMyBAQCBldmVudF9kaXNwYXRjaF9wb2xsIChz dHJ1Y3QgZXZlbnRfcG9vbCAqCiAgICAgICAgICAgICAgICAgICAgICAgICAvKiBzeXMgY2FsbCAq LwogICAgICAgICAgICAgICAgICAgICAgICAgY29udGludWU7CiAKLSAgICAgICAgICAgICAgICBm b3IgKGkgPSAwOyBpIDwgc2l6ZTsgaSsrKSB7CisgICAgICAgICAgICAgICAgaWYgKHVmZHNbMF0u cmV2ZW50cyAhPSAwICYmIGV2ZW50X3Bvb2wtPmJyZWFrZXJbMF0gPj0gMCkgeworICAgICAgICAg ICAgICAgICAgICAgICAgLyogSnVzdCByZWFkIGFsbCB0aGUganVuayBpbiB0aGUgYnJlYWtlciBw aXBlLiAqLworICAgICAgICAgICAgICAgICAgICAgICAgd2hpbGUgKHJlYWQoZXZlbnRfcG9vbC0+ YnJlYWtlclswXSwgJngsIDEpID4gMCkKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg OworICAgICAgICAgICAgICAgIH0KKworICAgICAgICAgICAgICAgIGZvciAoaSA9IDE7IGkgPCBz aXplOyBpKyspIHsKICAgICAgICAgICAgICAgICAgICAgICAgIGlmICghdWZkc1tpXS5yZXZlbnRz KQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb250aW51ZTsKIAo= ------=_Part_154259808_2047258858.1452307215166--