Date: Fri, 8 Jan 2016 21:40:15 -0500 (EST) From: Rick Macklem <rmacklem@uoguelph.ca> To: Jeff Darcy <jdarcy@redhat.com> Cc: Raghavendra G <raghavendra@gluster.com>, freebsd-fs <freebsd-fs@freebsd.org>, Hubbard Jordan <jkh@ixsystems.com>, Xavier Hernandez <xhernandez@datalab.es>, Gluster Devel <gluster-devel@gluster.org> Subject: Re: [Gluster-devel] FreeBSD port of GlusterFS racks up a lot of CPU usage Message-ID: <2013962695.154259810.1452307215168.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <1924941590.6473225.1452248249994.JavaMail.zimbra@redhat.com> References: <571237035.145690509.1451437960464.JavaMail.zimbra@uoguelph.ca> <20151230103152.GS13942@ndevos-x240.usersys.redhat.com> <2D8C2729-D556-479B-B4E2-66E1BB222F41@ixsystems.com> <1083933309.146084334.1451517977647.JavaMail.zimbra@uoguelph.ca> <CADRNtgStOg8UZfxNt-SzvvPf7d1J7CC_gi49ww3BbixU0Ey-rg@mail.gmail.com> <568F6D07.6070500@datalab.es> <CADRNtgRM17Eg3Z=LWifVNo=ai72dMiEVRKS3RwNfQ-dK7Pspew@mail.gmail.com> <1924941590.6473225.1452248249994.JavaMail.zimbra@redhat.com>
next in thread | previous in thread | raw e-mail | index | archive | help
------=_Part_154259808_2047258858.1452307215166 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Oops, I realized the last patch did a write(2) while holding a pthread_mutex. I've never used pthread_mutexes, but I suspect this isn't allowed. The attached updated patch delays the write() until after the pthread_mutex_unlock(). Sorry about the confusion, rick ----- Original Message ----- > > > I don't know anything about gluster's poll implementation so I may > > > be totally wrong, but would it be possible to use an eventfd (or a > > > pipe if eventfd is not supported) to signal the need to add more > > > file descriptors to the poll call ? > > > > > > > > > The poll call should listen on this new fd. When we need to change > > > the fd list, we should simply write to the eventfd or pipe from > > > another thread. This will cause the poll call to return and we will > > > be able to change the fd list without having a short timeout nor > > > having to decide on any trade-off. > > > > > > Thats a nice idea. Based on my understanding of why timeouts are being > > used, this approach can work. > > The own-thread code which preceded the current poll implementation did > something similar, using a pipe fd to be woken up for new *outgoing* > messages. That code still exists, and might provide some insight into > how to do this for the current poll code. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > ------=_Part_154259808_2047258858.1452307215166 Content-Type: text/x-patch; name=gluster-poll.patch Content-Disposition: attachment; filename=gluster-poll.patch Content-Transfer-Encoding: base64 LS0tIGdsdXN0ZXJmcy0zLjcuNi9saWJnbHVzdGVyZnMvc3JjL2V2ZW50LXBvbGwuYy5zYXYJMjAx Ni0wMS0wNiAxNTo1ODowMy41MjIyODYwMDAgLTA1MDAKKysrIGdsdXN0ZXJmcy0zLjcuNi9saWJn bHVzdGVyZnMvc3JjL2V2ZW50LXBvbGwuYwkyMDE2LTAxLTA4IDE4OjE0OjU3LjY1ODY1MjAwMCAt MDUwMApAQCAtMTgwLDYgKzE4MCwxNSBAQCBldmVudF9wb29sX25ld19wb2xsIChpbnQgY291bnQs IGludCBldmVuCiAgICAgICAgIHJldHVybiBldmVudF9wb29sOwogfQogCitzdGF0aWMgdm9pZAor ZXZlbnRfcG9vbF9jaGFuZ2VkIChzdHJ1Y3QgZXZlbnRfcG9vbCAqZXZlbnRfcG9vbCkKK3sKKwor ICAgICAgICAvKiBXcml0ZSBhIGJ5dGUgaW50byB0aGUgYnJlYWtlciBwaXBlIHRvIHdha2UgdXAg cG9sbCgpLiAqLworICAgICAgICBpZiAoZXZlbnRfcG9vbC0+YnJlYWtlclsxXSA+PSAwKQorICAg ICAgICAgICAgICAgIHdyaXRlKGV2ZW50X3Bvb2wtPmJyZWFrZXJbMV0sICJYIiwgMSk7Cit9CisK IAogc3RhdGljIGludAogZXZlbnRfcmVnaXN0ZXJfcG9sbCAoc3RydWN0IGV2ZW50X3Bvb2wgKmV2 ZW50X3Bvb2wsIGludCBmZCwKQEAgLTE4Nyw2ICsxOTYsNyBAQCBldmVudF9yZWdpc3Rlcl9wb2xs IChzdHJ1Y3QgZXZlbnRfcG9vbCAqCiAgICAgICAgICAgICAgICAgICAgICB2b2lkICpkYXRhLCBp bnQgcG9sbF9pbiwgaW50IHBvbGxfb3V0KQogewogICAgICAgICBpbnQgaWR4ID0gLTE7CisgICAg ICAgIGludCBjaGFuZ2VkID0gMDsKIAogICAgICAgICBHRl9WQUxJREFURV9PUl9HT1RPICgiZXZl bnQiLCBldmVudF9wb29sLCBvdXQpOwogCkBAIC0yNDUsMTAgKzI1NSwxMyBAQCBldmVudF9yZWdp c3Rlcl9wb2xsIChzdHJ1Y3QgZXZlbnRfcG9vbCAqCiAgICAgICAgICAgICAgICAgfQogCiAgICAg ICAgICAgICAgICAgZXZlbnRfcG9vbC0+Y2hhbmdlZCA9IDE7CisgICAgICAgICAgICAgICAgY2hh bmdlZCA9IDE7CiAKICAgICAgICAgfQogdW5sb2NrOgogICAgICAgICBwdGhyZWFkX211dGV4X3Vu bG9jayAoJmV2ZW50X3Bvb2wtPm11dGV4KTsKKyAgICAgICAgaWYgKGNoYW5nZWQgIT0gMCkKKyAg ICAgICAgICAgICAgICBldmVudF9wb29sX2NoYW5nZWQoZXZlbnRfcG9vbCk7CiAKIG91dDoKICAg ICAgICAgcmV0dXJuIGlkeDsKQEAgLTI1OSw2ICsyNzIsNyBAQCBzdGF0aWMgaW50CiBldmVudF91 bnJlZ2lzdGVyX3BvbGwgKHN0cnVjdCBldmVudF9wb29sICpldmVudF9wb29sLCBpbnQgZmQsIGlu dCBpZHhfaGludCkKIHsKICAgICAgICAgaW50IGlkeCA9IC0xOworICAgICAgICBpbnQgY2hhbmdl ZCA9IDA7CiAKICAgICAgICAgR0ZfVkFMSURBVEVfT1JfR09UTyAoImV2ZW50IiwgZXZlbnRfcG9v bCwgb3V0KTsKIApAQCAtMjc2LDkgKzI5MCwxMiBAQCBldmVudF91bnJlZ2lzdGVyX3BvbGwgKHN0 cnVjdCBldmVudF9wb29sCiAKICAgICAgICAgICAgICAgICBldmVudF9wb29sLT5yZWdbaWR4XSA9 ICBldmVudF9wb29sLT5yZWdbLS1ldmVudF9wb29sLT51c2VkXTsKICAgICAgICAgICAgICAgICBl dmVudF9wb29sLT5jaGFuZ2VkID0gMTsKKyAgICAgICAgICAgICAgICBjaGFuZ2VkID0gMTsKICAg ICAgICAgfQogdW5sb2NrOgogICAgICAgICBwdGhyZWFkX211dGV4X3VubG9jayAoJmV2ZW50X3Bv b2wtPm11dGV4KTsKKyAgICAgICAgaWYgKGNoYW5nZWQgIT0gMCkKKyAgICAgICAgICAgICAgICBl dmVudF9wb29sX2NoYW5nZWQoZXZlbnRfcG9vbCk7CiAKIG91dDoKICAgICAgICAgcmV0dXJuIGlk eDsKQEAgLTMwNCw2ICszMjEsNyBAQCBldmVudF9zZWxlY3Rfb25fcG9sbCAoc3RydWN0IGV2ZW50 X3Bvb2wgCiAgICAgICAgICAgICAgICAgICAgICAgaW50IHBvbGxfaW4sIGludCBwb2xsX291dCkK IHsKICAgICAgICAgaW50IGlkeCA9IC0xOworICAgICAgICBpbnQgY2hhbmdlZCA9IDA7CiAKICAg ICAgICAgR0ZfVkFMSURBVEVfT1JfR09UTyAoImV2ZW50IiwgZXZlbnRfcG9vbCwgb3V0KTsKIApA QCAtMzQ5LDExICszNjcsMTUgQEAgZXZlbnRfc2VsZWN0X29uX3BvbGwgKHN0cnVjdCBldmVudF9w b29sIAogICAgICAgICAgICAgICAgICAgICAgICAgYnJlYWs7CiAgICAgICAgICAgICAgICAgfQog Ci0gICAgICAgICAgICAgICAgaWYgKHBvbGxfaW4gKyBwb2xsX291dCA+IC0yKQorICAgICAgICAg ICAgICAgIGlmIChwb2xsX2luICsgcG9sbF9vdXQgPiAtMikgewogICAgICAgICAgICAgICAgICAg ICAgICAgZXZlbnRfcG9vbC0+Y2hhbmdlZCA9IDE7CisgICAgICAgICAgICAgICAgICAgICAgICBj aGFuZ2VkID0gMTsKKyAgICAgICAgICAgICAgICB9CiAgICAgICAgIH0KIHVubG9jazoKICAgICAg ICAgcHRocmVhZF9tdXRleF91bmxvY2sgKCZldmVudF9wb29sLT5tdXRleCk7CisgICAgICAgIGlm IChjaGFuZ2VkICE9IDApCisgICAgICAgICAgICAgICAgZXZlbnRfcG9vbF9jaGFuZ2VkKGV2ZW50 X3Bvb2wpOwogCiBvdXQ6CiAgICAgICAgIHJldHVybiBpZHg7CkBAIC00NDgsNiArNDcwLDcgQEAg ZXZlbnRfZGlzcGF0Y2hfcG9sbCAoc3RydWN0IGV2ZW50X3Bvb2wgKgogICAgICAgICBpbnQgICAg ICAgICAgICAgIHNpemUgPSAwOwogICAgICAgICBpbnQgICAgICAgICAgICAgIGkgPSAwOwogICAg ICAgICBpbnQgICAgICAgICAgICAgIHJldCA9IC0xOworICAgICAgICBjaGFyICAgICAgICAgICAg IHg7CiAKICAgICAgICAgR0ZfVkFMSURBVEVfT1JfR09UTyAoImV2ZW50IiwgZXZlbnRfcG9vbCwg b3V0KTsKIApAQCAtNDcyLDcgKzQ5NSw3IEBAIGV2ZW50X2Rpc3BhdGNoX3BvbGwgKHN0cnVjdCBl dmVudF9wb29sICoKICAgICAgICAgICAgICAgICBzaXplID0gZXZlbnRfZGlzcGF0Y2hfcG9sbF9y ZXNpemUgKGV2ZW50X3Bvb2wsIHVmZHMsIHNpemUpOwogICAgICAgICAgICAgICAgIHVmZHMgPSBl dmVudF9wb29sLT5ldmNhY2hlOwogCi0gICAgICAgICAgICAgICAgcmV0ID0gcG9sbCAodWZkcywg c2l6ZSwgMSk7CisgICAgICAgICAgICAgICAgcmV0ID0gcG9sbCAodWZkcywgc2l6ZSwgLTEpOwog CiAgICAgICAgICAgICAgICAgaWYgKHJldCA9PSAwKQogICAgICAgICAgICAgICAgICAgICAgICAg LyogdGltZW91dCAqLwpAQCAtNDgyLDcgKzUwNSwxMyBAQCBldmVudF9kaXNwYXRjaF9wb2xsIChz dHJ1Y3QgZXZlbnRfcG9vbCAqCiAgICAgICAgICAgICAgICAgICAgICAgICAvKiBzeXMgY2FsbCAq LwogICAgICAgICAgICAgICAgICAgICAgICAgY29udGludWU7CiAKLSAgICAgICAgICAgICAgICBm b3IgKGkgPSAwOyBpIDwgc2l6ZTsgaSsrKSB7CisgICAgICAgICAgICAgICAgaWYgKHVmZHNbMF0u cmV2ZW50cyAhPSAwICYmIGV2ZW50X3Bvb2wtPmJyZWFrZXJbMF0gPj0gMCkgeworICAgICAgICAg ICAgICAgICAgICAgICAgLyogSnVzdCByZWFkIGFsbCB0aGUganVuayBpbiB0aGUgYnJlYWtlciBw aXBlLiAqLworICAgICAgICAgICAgICAgICAgICAgICAgd2hpbGUgKHJlYWQoZXZlbnRfcG9vbC0+ YnJlYWtlclswXSwgJngsIDEpID4gMCkKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg OworICAgICAgICAgICAgICAgIH0KKworICAgICAgICAgICAgICAgIGZvciAoaSA9IDE7IGkgPCBz aXplOyBpKyspIHsKICAgICAgICAgICAgICAgICAgICAgICAgIGlmICghdWZkc1tpXS5yZXZlbnRz KQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb250aW51ZTsKIAo= ------=_Part_154259808_2047258858.1452307215166--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2013962695.154259810.1452307215168.JavaMail.zimbra>