From owner-freebsd-current@FreeBSD.ORG Thu Sep 12 04:59:19 2013 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id CE31EC57 for ; Thu, 12 Sep 2013 04:59:19 +0000 (UTC) (envelope-from hiren.panchasara@gmail.com) Received: from mail-ve0-x232.google.com (mail-ve0-x232.google.com [IPv6:2607:f8b0:400c:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8D5DF2F26 for ; Thu, 12 Sep 2013 04:59:19 +0000 (UTC) Received: by mail-ve0-f178.google.com with SMTP id jw12so6515947veb.37 for ; Wed, 11 Sep 2013 21:59:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=eFC95lXItiKnPBYjsnxGkgfrIzvc94Zh9vxQUFfoJQY=; b=QFUP8uirgfeDRCRHR/9YRYfhu/A9PBupIehzmh+P1Pt7w2mrqpRP6v5XjYqU185AYk 0uNZCkQSFkqENXvo9e7HmKADZAdHrf3NfDPcreB4wrOyvO2vQjgXoTOw4hEtrcw3aamm Ef90zUPnZ/n4upeUu2JtQMtdXKeu6PmCBqeqhh57k9m9OJqLuC8n19/YP4RdRoaARHQa RLE3xd4giR8rAvPqgiV6uu32FZD02H6QvirNM4LBZJX/4hHgkrn2S8GBGLY6tQuGHXwu VEtkpagANkArvE6MQ2gXqlZn/CeOwJvdpyY2U3j+qtdA7w9Px9rMv1F3TinpdJQN/0wi nk+g== MIME-Version: 1.0 X-Received: by 10.58.235.193 with SMTP id uo1mr4788535vec.6.1378961958656; Wed, 11 Sep 2013 21:59:18 -0700 (PDT) Received: by 10.58.173.101 with HTTP; Wed, 11 Sep 2013 21:59:18 -0700 (PDT) In-Reply-To: References: Date: Wed, 11 Sep 2013 21:59:18 -0700 Message-ID: Subject: Re: Intermittent hangs in vmem_periodic() From: hiren panchasara To: freebsd-current Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Sep 2013 04:59:19 -0000 On Wed, Sep 11, 2013 at 9:51 PM, hiren panchasara < hiren.panchasara@gmail.com> wrote: > I am seeing really weird system freeze for 2-3 seconds every minute or so. > > % uname -a > FreeBSD flymockour-l7.corp.yahoo.com 10.0-CURRENT FreeBSD 10.0-CURRENT #2 > r255423M: Mon Sep 9 12:29:22 PDT 2013 root@flymocko > ur-l7.corp.yahoo.com:/usr/obj/usr/home/hirenp/head/sys/GENERIC amd64 > > Simple comands like "man top" hangs and this is what I see in ctrl+T > % man top > load: 1.59 cmd: zcat 76574 [*kmem arena] 2.16r 0.00u 1.82s 16% 2000k > load: 1.70 cmd: zcat 76574 [*kmem arena] 7.19r 0.00u 1.82s 7% 2000k > load: 1.89 cmd: zcat 76574 [*kmem arena] 12.58r 0.00u 1.82s 3% 2000k > load: 1.90 cmd: zcat 76574 [*kmem arena] 14.58r 0.00u 1.82s 2% 2000k > > I see similar things while trying to "exit" a terminal window. > > I see following in pmcstat: > > PMC: [INSTR_RETIRED_ANY] Samples: 117647 (100.0%) , 6885 unresolved > > %SAMP CALLTREE > 75.9 fork_exit@kernel taskqueue_thread_loop taskqueue_run_locked > vmem_periodic(75.9%) > At any point in time whether system is misbehaving or not, I see this on top of the graph. Which seems wrong. Providing a few samples over the time: PMC: [INSTR_RETIRED_ANY] Samples: 97253 (100.0%) , 6080 unresolved %SAMP CALLTREE 78.1 fork_exit@kernel taskqueue_thread_loop taskqueue_run_locked vmem_periodic(78.1%) 1.0 bsearch@libc.so.7(1.0%) 0.7 0xe7b661@libnvidia-glcore.s clock_gettime@libc.so.7 0x128c6e 0x128a80(0.6%) 0.2 amd64_syscall@kernel sys_select kern_select 0.2 lapic_handle_timer@kernel timercb handleevents hardclock_cnt pmc_hook_handler@hwpmc.ko pmc_process_samples 0.0 amd64_syscall@kernel sys_poll 0.0 free@libc.so.7 0.0 0x212ff56@libxul.so 0x2130835 0x212f907 0x212f1e7 0x212d2fa 0x22b6777 0x22b6478 PMC: [INSTR_RETIRED_ANY] Samples: 35320 (100.0%) , 3663 unresolved %SAMP CALLTREE 32.4 fork_exit@kernel taskqueue_thread_loop taskqueue_run_locked vmem_periodic(32.4%) 1.6 free@libc.so.7 0x9ad75 0x9a597(1.4%) 1.3 intr_event_execute_handlers@kernel hdac_intr_handler hdaa_stream_intr chn_intr chn_intr_locked sndbuf_feed feed_mixer_feed z_resampler_feed z_feed_linear_S16LE(1.0%) 1.3 hdac_intr_handler@kernel hdaa_stream_intr chn_intr chn_intr_locked sndbuf_feed feed_mixer_feed z_resampler_feed z_feed_linear_S16LE(1.0%) 1.2 amd64_syscall@kernel sys___sysctl userland_sysctl sysctl_root cpufreq_curr_sysctl cf_set_method est_set DELAY(0.6%) get_tsc(0.5%) 1.1 bsearch@libc.so.7(1.1%) PMC: [INSTR_RETIRED_ANY] Samples: 118799 (100.0%) , 7859 unresolved %SAMP CALLTREE 70.5 fork_exit@kernel taskqueue_thread_loop taskqueue_run_locked vmem_periodic(70.5%) 1.1 bsearch@libc.so.7(1.1%) 0.2 lapic_handle_timer@kernel timercb handleevents hardclock_cnt pmc_hook_handler@hwpmc.ko pmc_process_samples 0.1 free@libc.so.7 0.1 0xe7b661@libnvidia-glcore.s clock_gettime@libc.so.7 0.1 amd64_syscall@kernel sys_select kern_select 0.1 __realloc@libc.so.7 0.1 amd64_syscall@kernel sys_poll cheers, Hiren > > I am not running anything funny and I do not see anything weird in "top" > too. > > Any help is appreciated. > > cheers, > Hiren >