From owner-freebsd-jail@freebsd.org Mon May 16 13:08:44 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1EFD7B3CF5B for ; Mon, 16 May 2016 13:08:44 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D9ACF1239 for ; Mon, 16 May 2016 13:08:43 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id E01E725D37C7; Mon, 16 May 2016 13:08:40 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 3F359D1F8A7; Mon, 16 May 2016 13:08:40 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id RbPs8hP7kxxL; Mon, 16 May 2016 13:08:38 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4410:50f4:9329:a995:3c2a] (unknown [IPv6:fde9:577b:c1a9:4410:50f4:9329:a995:3c2a]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 3627DD1F8A5; Mon, 16 May 2016 13:08:37 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: Unresponsive jails issues From: "Bjoern A. Zeeb" In-Reply-To: <6beab349-73bb-7159-cd81-443e115b687a@gjunka.com> Date: Mon, 16 May 2016 13:08:18 +0000 Cc: freebsd-jail@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <7ACDBC85-5B17-4695-8DAD-BCC48817EEBF@lists.zabbadoz.net> References: <6beab349-73bb-7159-cd81-443e115b687a@gjunka.com> To: Grzegorz Junka X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 May 2016 13:08:44 -0000 > On 16 May 2016, at 12:55 , Grzegorz Junka wrote: >=20 > I have a server running 13 jails for various system services. Recently = I added two jails to run simple go applications for testing. They open a = network socket and nginx, which is in another jail, and which round = robin balances requests to them. I mention that because it may be = related, however not necessarily because it was happening earlier. >=20 > The problem is that every 2-3 days jails in my servers stop = responding. "jexec jailname tcsh" hangs forever, "service jail stop = jailname" hangs forever as well. "top" doesn't show anything suspicious. = I can login through SSH to the main server fine. I don't login to jails = through SSH so I can't check but it seems that when that happens they = stop responding because the services that are running in them stop too = (e.g. web server, imap, ...). I tried to "kill -9" the "jexec" process = that hangs but that doesn't work. >=20 > My first question is what evidence should I gather when that happens = so that I can investigate the issue later on after the server is = restarted? >=20 > And the second question, any idea why that might be happening in the = first place? >=20 > I am running FreeBSD 10.3 AMD64 updated from 10.2 a couple of weeks = ago. If you can log into the base system and issue commands there; try to = see what procstat (-k) thinks about various jailed processes. You could = also check ps axl for the WCHAN and see if anything suspicious shows up. /bz