From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 5 01:54:13 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5AEE1106564A for ; Thu, 5 Apr 2012 01:54:13 +0000 (UTC) (envelope-from sushanth_rai@yahoo.com) Received: from nm24.bullet.mail.sp2.yahoo.com (nm24.bullet.mail.sp2.yahoo.com [98.139.91.94]) by mx1.freebsd.org (Postfix) with SMTP id 196A28FC08 for ; Thu, 5 Apr 2012 01:54:13 +0000 (UTC) Received: from [98.139.91.66] by nm24.bullet.mail.sp2.yahoo.com with NNFMP; 05 Apr 2012 01:54:07 -0000 Received: from [98.139.44.86] by tm6.bullet.mail.sp2.yahoo.com with NNFMP; 05 Apr 2012 01:54:07 -0000 Received: from [127.0.0.1] by omp1023.access.mail.sp2.yahoo.com with NNFMP; 05 Apr 2012 01:54:07 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 335107.92701.bm@omp1023.access.mail.sp2.yahoo.com Received: (qmail 63198 invoked by uid 60001); 5 Apr 2012 01:54:07 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1333590847; bh=LNg9qmn432+R6Ufk2u7Wc/MpS6a3zEleE5cn5I4g/bI=; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:MIME-Version:Content-Type; b=FTIy0/KlqagiVS7KY6aZ0z99YPHGOrVtnsNdHgv6UxJGkzqkSz3fwRLQF+RMdpoI0zPoJI2jfHKYYptnPSRnjEkLJAO7h5KFV0gfKKqICcNvOp32ayqICgeZR+7avWikqYQTOrX/F+UNWcLb+PWaLx6uIlfxSvyVif76IdHpH2w= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:MIME-Version:Content-Type; b=WCvdYyCrvkJIXV9WP3dJJYX1O55UBcUIMsJ0aiXSngKdMPcMgJZF0QCyslXtvS8J5dQ30DVPnDsDoqWwBaQ1Fc5iUApo1KQjn6NMmeAbL27Smyuy2Ibhzcm9pCjbDg9m3YL5iOEdZULQeiXNBsKx8xi3ku+9D7Wrsj3ftbUvAws=; X-YMail-OSG: nwEIMXMVM1kmgWH9wvp7Oggn8tcyuOSAjeP6YUE6lplwrlp hafuRuMTCoy4N2_IZXcGGLwcsd_zzjS9aOEcBxLi2zNMM.MTD24f2kVDRKZg iq6eSBfx7jt.EfDlP8IME4LhMhxOB_IKP0esPnF.Lem0yKZVS9r68D.zoEqO Xs_RT81D4gNaRuzhd7J3o496EsU4YOo_d2QD5bA_f6rSkX0H1fnwfNj7b5tC hiLmz14rdhfJiGpgSIZ1j3UR11__YOmwB_19pKApAvcWEvxFmUGkBEig7IJk YESYRttlqgEmzkyyMx61L6bpQ127XN2amgJDlUzp9yJQE_sxTR9ruvOXdDWN 0VMMRqg4kYdgcuTvb7j2GtIzBhc.xds91PZSSRGw1xIbz1tIAIsJfIHa9Otw xrGmgbq9CueIcK7ajY4Dgyxe64wsm.NGq6upBNyuZ4istRPhMl7exEinvlY6 xxvfTVEw- Received: from [209.119.38.67] by web180011.mail.gq1.yahoo.com via HTTP; Wed, 04 Apr 2012 18:54:06 PDT X-Mailer: YahooMailClassic/15.0.5 YahooMailWebService/0.8.117.340979 Message-ID: <1333590846.58474.YahooMailClassic@web180011.mail.gq1.yahoo.com> Date: Wed, 4 Apr 2012 18:54:06 -0700 (PDT) From: Sushanth Rai To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Startvation of realtime piority threads X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Apr 2012 01:54:13 -0000 I have a multithreaded user space program that basically runs at realtime priority. Synchronization between threads are done using spinlock. When running this program on a SMP system under heavy memory pressure I see that thread holding the spinlock is starved out of cpu. The cpus are effectively consumed by other threads that are spinning for lock to become available. After instrumenting the kernel a little bit what I found was that under memory pressure, when the user thread holding the spinlock traps into the kernel due to page fault, that thread sleeps until the free pages are available. The thread sleeps PUSER priority (within vm_waitpfault()). When it is ready to run, it is queued at PUSER priority even thought it's base priority is realtime. The other siblings threads that are spinning at realtime priority to acquire the spinlock starves the owner of spinlock. I was wondering if the sleep in vm_waitpfault() should be a MAX(td_user_pri, PUSER) instead of just PUSER. I'm running on 7.2 and it looks like this logic is the same in the trunk. Thanks, Sushanth