From owner-freebsd-stable@FreeBSD.ORG Mon Nov 26 07:43:56 2012 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0C9734C0 for ; Mon, 26 Nov 2012 07:43:56 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) by mx1.freebsd.org (Postfix) with ESMTP id D1F2F8FC08 for ; Mon, 26 Nov 2012 07:43:55 +0000 (UTC) Received: by mail-pa0-f54.google.com with SMTP id bi5so2133758pad.13 for ; Sun, 25 Nov 2012 23:43:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=FqzILQQSBXuVYpOoiGdmBtfidTk9oyL+QNiH2aADJ7g=; b=pOXaIR91OBye9wI8kDdVv5q/dkWP7PtcgH9+e0z3yBH14/3gPP/rmd2CZF6wZZhEas xYoAuEWJ0SZ+27uBGKS9SmQKK4FhejMSQdxGIBYm4xqUtQdR2zsibYPeD0tjuxfxo8Cw K9v49QlrnBJPF2RLL6EVGL9MOPZ/Z8rpSpcKaTlA0IaczLJ4ecyqMnbe1S3RiWPJbz8o ekj5jWybdSJW0cDNTmFW8bS15MljYSV/LGiG9nYi9g5QzlkEAe0lXc01RjNECZp6noWy j0fodKH8fAoA4YZLB2So+PEqNkFla0/GmGENZytG1f9f6R532dzOt5/HRoXU59ixwEG5 YThA== MIME-Version: 1.0 Received: by 10.68.248.10 with SMTP id yi10mr35192157pbc.39.1353915835463; Sun, 25 Nov 2012 23:43:55 -0800 (PST) Received: by 10.67.2.4 with HTTP; Sun, 25 Nov 2012 23:43:55 -0800 (PST) In-Reply-To: <20121126073810.GB90425@gmail.com> References: <20121126073810.GB90425@gmail.com> Date: Mon, 26 Nov 2012 09:43:55 +0200 Message-ID: Subject: Re: hastctl hang From: Mikolaj Golub To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Daisuke Aoyama X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Nov 2012 07:43:56 -0000 Sorry, the message went privately to Daisuke, which was not my intention. ---------- Forwarded message ---------- From: Mikolaj Golub Date: Mon, Nov 26, 2012 at 9:38 AM Subject: Re: hastctl hang To: Daisuke Aoyama On Mon, Nov 26, 2012 at 01:17:46AM +0900, Daisuke Aoyama wrote: > Hello, > > I'm trying to integrate HAST to NAS4Free (FreeBSD 9.1-RC3). > Now I have created version 9.1.0.1.531. > http://sourceforge.net/projects/nas4free/files/NAS4Free-9.1.0.1/9.1.0.1.531/ > > Basic CARP + HAST + iSCSI target setup can be done, but very frequently I > get hastctl hang when called: > > /sbin/hastctl status > /sbin/hastctl dump > > Is it better for this method not to call from a script? > or somthing wrong to use it? Normally it is ok to use hastctl for scripting. Do you have it hang forever of just for a few seconds? Usually hanged hastctl means that hastd master process is waiting for its worker (either its response or exit). Could you provide logs from both master ans secondary? Also you might want to run hastd with -d to make it more verbose. > Also, I don't know how to detect an error of writing to local device from > hastd. > Does anyone know about it? Currently only by monitoring logs. It looks like a good idea to add error counters to hastctl statistics output... > Thanks, > Daisuke Aoyama > > -- the procstat shows like this: > [root@nas4free-nodeb /tmp]# procstat -ka|grep hast > 11668 100069 hastd - mi_switch > sleepq_catch_signals sleepq_wait_sig _sleep kern_wait sys_wait4 > amd64_syscall Xfast_syscall > 17981 100406 hastd - mi_switch > sleepq_catch_signals sleepq_wait_sig _sleep do_wait > __umtx_op_wait_uint_private amd64_syscall Xfast_syscall > 17981 100559 hastd - mi_switch > sleepq_catch_signals sleepq_wait_sig _sleep soreceive_generic kern_recvit > recvit sys_recvfrom amd64_syscall Xfast_syscall > 17981 100560 hastd - mi_switch > sleepq_catch_signals sleepq_wait_sig _sleep soreceive_generic kern_recvit > recvit sys_recvfrom amd64_syscall Xfast_syscall > 17981 100561 hastd - mi_switch > sleepq_catch_signals sleepq_wait_sig _sleep do_wait > __umtx_op_wait_uint_private amd64_syscall Xfast_syscall > 17984 100078 hastd - mi_switch > sleepq_catch_signals sleepq_wait_sig _sleep do_wait > __umtx_op_wait_uint_private amd64_syscall Xfast_syscall > 17984 100562 hastd - mi_switch > sleepq_catch_signals sleepq_wait_sig _sleep soreceive_generic kern_recvit > recvit sys_recvfrom amd64_syscall Xfast_syscall > 17984 100563 hastd - mi_switch > sleepq_catch_signals sleepq_wait_sig _sleep soreceive_generic kern_recvit > recvit sys_recvfrom amd64_syscall Xfast_syscall > 17984 100564 hastd - mi_switch > sleepq_catch_signals sleepq_wait_sig _sleep do_wait > __umtx_op_wait_uint_private amd64_syscall Xfast_syscall > 18218 100145 hastctl - mi_switch > sleepq_catch_signals sleepq_wait_sig _sleep soreceive_generic kern_recvit > recvit sys_recvfrom amd64_syscall Xfast_syscall > > [root@nas4free-nodeb /tmp]# procstat -ta|grep hast > 11668 100069 hastd - 0 120 sleep wait > 17979 100557 hastd - 2 120 sleep g_waitid Strange, I don't see 17979 process in procstat -k output. Again, the logs might be helpful here. > 17981 100406 hastd - 2 120 sleep uwait > 17981 100559 hastd - 0 120 sleep sbwait > 17981 100560 hastd - 0 120 sleep sbwait > 17981 100561 hastd - 1 120 sleep uwait > 17984 100078 hastd - 2 121 sleep uwait > 17984 100562 hastd - 3 120 sleep sbwait > 17984 100563 hastd - 2 120 sleep sbwait > 17984 100564 hastd - 1 121 sleep uwait > 18218 100145 hastctl - 2 152 sleep sbwait > -- the procstat shows like this: > > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" -- Mikolaj Golub