From owner-freebsd-stable@FreeBSD.ORG Tue Aug 19 16:37:04 2014 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 473F0F6B for ; Tue, 19 Aug 2014 16:37:04 +0000 (UTC) Received: from mail.intermedix.com (mail.epbs.com [66.210.191.9]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "Barracuda/emailAddress=sales@barracuda.com", Issuer "Barracuda/emailAddress=sales@barracuda.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 06A3433E9 for ; Tue, 19 Aug 2014 16:37:03 +0000 (UTC) X-ASG-Debug-ID: 1408466221-049956373840dde0001-BIHDGU Received: from mailgate01.corp.okcyok1.priv.intermedix.com (mailgate01.epbs.com [10.130.4.35]) by mail.intermedix.com with ESMTP id asyuvViMUGuqnf2p; Tue, 19 Aug 2014 11:37:01 -0500 (CDT) X-Barracuda-Envelope-From: Steve.Polyack@intermedix.com X-ASG-Whitelist: Client X-WSS-ID: 0NAKBHM-02-4MC-02 X-M-MSG: Received: from exchange01.epbs.com (exchange01.okcyok0.priv.intermedix.com [192.168.25.157]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mailgate01.corp.okcyok1.priv.intermedix.com (Postfix) with ESMTPS id 20CB031A166; Tue, 19 Aug 2014 11:36:57 -0500 (CDT) Received: from EXCHANGE03.epbs.com ([0000:0000:0000:0000:0000:0000:0.0.0.1]) by exchange01.epbs.com ([192.168.25.157]) with mapi; Tue, 19 Aug 2014 11:37:00 -0500 From: "Polyack, Steve" To: Alan Cox , "freebsd-stable@freebsd.org" Date: Tue, 19 Aug 2014 11:36:58 -0500 Subject: RE: vmdaemon CPU usage and poor performance in 10.0-RELEASE Thread-Topic: vmdaemon CPU usage and poor performance in 10.0-RELEASE X-ASG-Orig-Subj: RE: vmdaemon CPU usage and poor performance in 10.0-RELEASE Thread-Index: Ac+7MMhjygcYL2iOSAKl5ishegpU1wAmhXBA Message-ID: <4D557EC7CC2A544AA7C1A3B9CBA2B3672609CF28E5@exchange03.epbs.com> References: <4D557EC7CC2A544AA7C1A3B9CBA2B36726098846B4@exchange03.epbs.com> <20140813152522.GI9400@home.opsec.eu> <4D557EC7CC2A544AA7C1A3B9CBA2B36726098847AF@exchange03.epbs.com> <4D557EC7CC2A544AA7C1A3B9CBA2B3672609BBA3C4@exchange03.epbs.com> <53F24E5B.1010809@rice.edu> <4D557EC7CC2A544AA7C1A3B9CBA2B3672609BBA64F@exchange03.epbs.com> <53F2790C.20703@rice.edu> In-Reply-To: <53F2790C.20703@rice.edu> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Barracuda-Connect: mailgate01.epbs.com[10.130.4.35] X-Barracuda-Start-Time: 1408466221 X-Barracuda-URL: http://192.168.25.21:8000/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at intermedix.com X-Barracuda-BRTS-Status: 1 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Aug 2014 16:37:04 -0000 > -----Original Message----- > From: owner-freebsd-stable@freebsd.org [mailto:owner-freebsd- > stable@freebsd.org] On Behalf Of Alan Cox > Sent: Monday, August 18, 2014 6:07 PM > To: freebsd-stable@freebsd.org > Subject: Re: vmdaemon CPU usage and poor performance in 10.0-RELEASE >=20 > On 08/18/2014 16:29, Polyack, Steve wrote: > >> -----Original Message----- > >> From: owner-freebsd-stable@freebsd.org [mailto:owner-freebsd- > >> stable@freebsd.org] On Behalf Of Alan Cox > >> Sent: Monday, August 18, 2014 3:05 PM > >> To: freebsd-stable@freebsd.org > >> Subject: Re: vmdaemon CPU usage and poor performance in 10.0-RELEASE > >> > >> On 08/18/2014 13:42, Polyack, Steve wrote: > >>> Excuse my poorly formatted reply at the moment, but this seems to > have > >> fixed our problems. I'm going to update the bug report with a note. > >>> Thanks Alan! > >> You're welcome. And, thanks for letting me know of the outcome. > >> > > Actually, I may have spoken too soon, as it looks like we're seeing > vmdaemon tying up the system again: > > root 6 100.0 0.0 0 16 - DL Wed04PM = 4:37.95 [vmdaemon] > > > > Is there anything I can check to help narrow down what may be the > problem? KTrace/truss on the "process" doesn't give any information, I > suppose because it's actually a kernel thread. >=20 > Can you provide the full output of top? Is there anything unusual about > the hardware or software configuration? This may have just been a fluke (maybe NFS caching the old vm_pageout.c dur= ing the first source build). We've rebuilt and are monitoring it now. The hardware consists of a few Dell PowerEdge R720xd servers with 256GB of = RAM and array of SSDs (no ZFS). 64GB is dedicated to postgres shared_buffe= rs right now. FreeBSD 10, PostgreSQL 9.3, Slony-I v2.2.2, and redis-2.8.11 = are all in use here. I can't say that anything is unusual about the config= uration. > > For the "patch" we simply stole vm_pageout.c from r265945. > > > > Steve > > > >>>> -----Original Message----- > >>>> From: owner-freebsd-stable@freebsd.org [mailto:owner-freebsd- > >>>> stable@freebsd.org] On Behalf Of Alan Cox > >>>> Sent: Wednesday, August 13, 2014 12:14 PM > >>>> To: Polyack, Steve > >>>> Cc: Kurt Jaeger; freebsd-stable@freebsd.org > >>>> Subject: Re: vmdaemon CPU usage and poor performance in 10.0- > RELEASE > >>>> > >>>> On Wed, Aug 13, 2014 at 10:42 AM, Polyack, Steve < > >>>> Steve.Polyack@intermedix.com> wrote: > >>>> > >>>>>> -----Original Message----- > >>>>>> From: Kurt Jaeger [mailto:lists@opsec.eu] > >>>>>> Sent: Wednesday, August 13, 2014 11:25 AM > >>>>>> > >>>>>> Hi! > >>>>>> > >>>>>>> We have a handful of database servers running FreeBSD 10.0- > RELEASE > >>>>>>> and PostgreSQL 9.3.4. The servers have 128GB or 256GB of RAM. > >>>>>> Are you aware of the recent work on that topic ? > >>>>>> > >>>>>> https://www.freebsd.org/news/status/report-2014-04-2014- > >>>>>> 06.html#PostgreSQL-Performance-Improvements > >>>>>> > >>>>>> Maybe kib@ knows more about this ? > >>>>>> > >>>>> I've recently read over this and some other posts, but they all see= m to > >>>>> center around poor postgres performance. In our case at least, som= e > >> light > >>>>> to medium usage of postgres generally makes the entire system > >> unusable. > >>>>> The patches & documents linked there also all seem to be for - > CURRENT, > >>>>> which we aren't running. We're not too keen on the idea of using > >>>> CURRENT > >>>>> in production, either. We're planning on testing 10-STABLE, but I = was > >> just > >>>>> hoping to gain some insight into what the problem may be and > whether > >>>> recent > >>>>> commits to vmdaemon code in the -STABLE tree may have a positive > >> effect > >>>> on > >>>>> what we've seen. > >>>>> > >>>>> > >>>> There is a good chance that your problem is fixed by r265945. > >>>> _______________________________________________ > >>>> freebsd-stable@freebsd.org mailing list > >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable > >>>> To unsubscribe, send any mail to "freebsd-stable- > >>>> unsubscribe@freebsd.org" > >>> _______________________________________________ > >>> freebsd-stable@freebsd.org mailing list > >>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable > >>> To unsubscribe, send any mail to "freebsd-stable- > >> unsubscribe@freebsd.org" > >> _______________________________________________ > >> freebsd-stable@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-stable > >> To unsubscribe, send any mail to "freebsd-stable- > >> unsubscribe@freebsd.org" > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable- > unsubscribe@freebsd.org" > > >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable- > unsubscribe@freebsd.org"