From owner-freebsd-stable@FreeBSD.ORG Fri Dec 26 13:23:05 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6497F1065675 for ; Fri, 26 Dec 2008 13:23:05 +0000 (UTC) (envelope-from pluknet@gmail.com) Received: from yw-out-2324.google.com (yw-out-2324.google.com [74.125.46.29]) by mx1.freebsd.org (Postfix) with ESMTP id 1E2308FC19 for ; Fri, 26 Dec 2008 13:23:04 +0000 (UTC) (envelope-from pluknet@gmail.com) Received: by yw-out-2324.google.com with SMTP id 9so2015824ywe.13 for ; Fri, 26 Dec 2008 05:23:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=7Nd5jPSn9W5fnR/nUcgPLbqaIFwcqvp7q0iFE6XpY7Q=; b=kSiLJ9EAYJveSIxktS0kRDdJ3uuX9IkgDA7usOBmVHTu8kKYyYoE75aq7/OtjNVUuO vkjl0k07aW0LGcmdt1VnVioeMwbhu5rTwNKP0dVJk1EZRJwWHGLXSMyFJGFRDQQAXfP1 l/Nqc4F9M6iszI3OlU4GnAPB/Ne3HkYnbNcOs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=Oj7zTS6To/OZBiwAXYk4tcLlHE0UvF4//23LovhFF1ZkDfJq+ysEY4hy5l2zVzexZj P9XLkKtpQMEhHzEAa3OalkArdVGp23bGqppaZuQI1XVmysBdSucNNmCZNfDVVLI3HAAL f5kFnsxrn7Yfk7aFoa3oq20avW4ds1jV7Mtrs= Received: by 10.90.120.19 with SMTP id s19mr5321076agc.29.1230297784252; Fri, 26 Dec 2008 05:23:04 -0800 (PST) Received: by 10.90.73.9 with HTTP; Fri, 26 Dec 2008 05:23:04 -0800 (PST) Message-ID: Date: Fri, 26 Dec 2008 16:23:04 +0300 From: pluknet To: "FreeBSD Stable Mailing List" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: Subject: Re: process stuck in vmopar X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Dec 2008 13:23:05 -0000 Hi again! 2008/12/24 pluknet : > 2008/12/24 pluknet : >> 2008/12/24 pluknet : >>> 2008/12/24 pluknet : >>>> Server version: Apache/2.2.11 (Unix) built from sources. >>>> >>>> After issuing kill -9 process stuck in vmopar state forever. >>>> aaa301 2313 0.0 0.0 0 8 ?? DE 3:10PM 0:00.01 >>>> /home/aaa301/myb vmopar >>>> >>>> System: FreeBSD 6.2 i386. >>>> >>> >>> One important note. >>> Kernel is built with options QUOTA, and this problem triggered >>> only when this user is overquoted (usage > quota and limit). >> >> A bit later various processes begin to stuck in "ufs" state. > > backtrace of process that stucks in vmopar: > > db> bt 1385 > Tracing pid 1385 tid 100181 td 0xc6c19960 > sched_switch(c6c19960,0,1) at sched_switch+0x15b > mi_switch(1,0) at mi_switch+0x270 > sleepq_switch(c2954ec8,c0a3d0a0,0,c094c4eb,211,...) at sleepq_switch+0xc1 > sleepq_wait(c2954ec8,0,c096897c,709,c096897c,...) at sleepq_wait+0x46 > msleep(c2954ec8,c0ab8e80,244,c0968ca9,0,c9030444,0,c0968eb0,200,c2954ec8,82) > at > msleep+0x279 > vm_page_sleep_if_busy(c2954ec8,1,c0968ca9) at vm_page_sleep_if_busy+0x7c > vm_object_page_remove(c9030444,4,0,8000,0,0) at vm_object_page_remove+0xf9 > vnode_pager_setsize(c903c000,4000,0) at vnode_pager_setsize+0xbd > ffs_write(f734a78c) at ffs_write+0x264 > VOP_WRITE_APV(c0a09b00,f734a78c) at VOP_WRITE_APV+0x112 > vnode_pager_generic_putpages(c903c000,f734a8d0,9000,5,f734a860,...) at > vnode_pag > er_generic_putpages+0x1ef > vop_stdputpages(f734a814) at vop_stdputpages+0x1a > VOP_PUTPAGES_APV(c0a09b00,f734a814) at VOP_PUTPAGES_APV+0x8c > vnode_pager_putpages(c9030444,f734a8d0,9,5,f734a860) at > vnode_pager_putpages+0x7 > e > vm_pageout_flush(f734a8d0,9,5,0,0,...) at vm_pageout_flush+0x112 > vm_object_page_collect_flush(c9030444,c29505a8,251,5,4a,...) at > vm_object_page_c > ollect_flush+0x2a0 > vm_object_page_clean(c9030444,0,0,0,0,...) at vm_object_page_clean+0x184 > vm_object_terminate(c9030444) at vm_object_terminate+0x60 > vnode_destroy_vobject(c903c000,c6973500,f734aab8,c6c19960,0,...) at > vnode_destro > y_vobject+0x39 > ufs_reclaim(f734aab8) at ufs_reclaim+0x46 > VOP_RECLAIM_APV(c0a09b00,f734aab8) at VOP_RECLAIM_APV+0x7e > vgonel(c903c000) at vgonel+0x12d > vrecycle(c903c000,c6c19960) at vrecycle+0x38 > ufs_inactive(f734ab40) at ufs_inactive+0x2af > VOP_INACTIVE_APV(c0a09b00,f734ab40) at VOP_INACTIVE_APV+0x7e > vinactive(c903c000,c6c19960) at vinactive+0x72 > vrele(c903c000,c9030444,0,c096897c,1a2,...) at vrele+0x14a > vm_object_vndeallocate(c9030444) at vm_object_vndeallocate+0xc0 > vm_object_deallocate(c9030444,c9030444,0,c0968016,8e7) at > vm_object_deallocate+0 > xb3 > vm_map_entry_delete(c722a000,c7194bf4,f734ac20,c081ca37,c722a000,...) > at vm_map_ > entry_delete+0x130 > vm_map_delete(c722a000,0,bfc00000) at vm_map_delete+0x18f > vmspace_exit(c6c19960,c0a4bde0,0,c09463ea,125,...) at vmspace_exit+0xd5 > exit1(c6c19960,9,2831a4b4,c6c19960,c7234000,...) at exit1+0x496 > sigexit(c6c19960,9,c7234aa8,0,c09499bc,...) at sigexit+0xdf > postsig(9) at postsig+0x160 > ast(f734ad38) at ast+0x35e > doreti_ast() at doreti_ast+0x17 > > db> show alllock > Process 1385 (httpd) thread 0xc6c19960 (100181) > exclusive sx user map r = 0 (0xc722a044) locked @ > /usr/src/sys_uvmem_uip.6.2_RELEASE/vm/vm_map.c:307 > Today I found some interesting details how to reproduce my problem. Stucking apache2.x in "vmopar" with subsequent stuck of other various processes in "ufs" is only triggered with those options enabled in php.ini: extension="xcache.so" xcache.size=64M xcache.count=8 xcache.slot=64K xcache.var_size=64M xcache.var_count=8 xcache.var_slots=64K xcache.mmap_path=/tmp/xcache Perhaps the problem is related to mmap vs threads interaction. Any thoughts? > -- > wbr, > pluknet >