From owner-freebsd-questions@FreeBSD.ORG Sun Oct 12 07:41:25 2014 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 636F9F8B for ; Sun, 12 Oct 2014 07:41:25 +0000 (UTC) Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com [IPv6:2a00:1450:400c:c05::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F2A961CA for ; Sun, 12 Oct 2014 07:41:24 +0000 (UTC) Received: by mail-wi0-f179.google.com with SMTP id d1so5138476wiv.6 for ; Sun, 12 Oct 2014 00:41:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Tcz14wyqzDsB8SgdYVdOXy7RuXWvXlXQ8LnBqkKcOK4=; b=V6w6Jzfhrr4Hu1rJoqcVZIdH7avflibDJ/8AIABBYTZ+4dNKvIHKA5xozZPqvwjx0h uX1I5V6sMZjNf+6Dn6H2KRAbry30+/DX3OMmIg9zRyZELKIQdGumFh52uGAv+UqZmPiz LLEKGXkKfN5clw2wKEw4jEjYfJEk8aj73FMBRcnM2XMl/DvuN9BSKMYyiwBUzev2hYJf +qA/AFVFySCUH7bIsLOq3Pb+G1TBRCpH8XtC5fT2VODcCnW7SR7mERBFf6U5DwUoaxMF 4JFlT4exljUVp9Tb91z62frK+WQLAykYQRGjtTzURu5WWP4mjvq4CjwOU4WolCFPA+Et rrUA== MIME-Version: 1.0 X-Received: by 10.180.20.46 with SMTP id k14mr13941569wie.26.1413099683215; Sun, 12 Oct 2014 00:41:23 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.216.106.136 with HTTP; Sun, 12 Oct 2014 00:41:23 -0700 (PDT) In-Reply-To: <1412732931033.813626ca@Nodemailer> References: <1412732931033.813626ca@Nodemailer> Date: Sun, 12 Oct 2014 00:41:23 -0700 X-Google-Sender-Auth: bav3ZxYdoh1AzxGSQIeWDwdKots Message-ID: Subject: Re: Next Steps to Debug ZFS Hang? From: Adrian Chadd To: Nick Sivo Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Questions X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Oct 2014 07:41:25 -0000 Hi! A bunch of ZFS hangs were found / fixed in FreeBSD-HEAD and I -think- backported to FreeBSD-10. I don't know if they've been backported to -9. Certainly not to 9.2; I think I found / reported some after 9.2 was released. In the kernel debugger (ddb), you can try "show allproc" to get a list of procs. https://www.freebsd.org/doc/en/books/developers-handbook/kerneldebug-online-ddb.html for more information. Why can't you get crashdumps? are you able to: * update to freebsd-9-stable? * .. if you can, update to freebsd-10-stable? 10.1 is about to be released soon. You could try "procstat -ta" to see the threads running. Do it as root to see all the threads. TDNAME is the thread name; WCHAN is what's important to figure out why it's sleeping. I hope this helps! -a On 7 October 2014 18:48, Nick Sivo wrote: > Hello, > > > I've been having trouble with ZFS on my server. For the most part it works splendidly, but occasionally I'll experience permanent hangs. > > > For example, right now on one of my ZFS filesystems (the others are fine), I can read, write, and stat files, but if I run ls in any directory, ls and the terminal will hang. CTRL-C, and kill -9 can't kill it: > > > In top: > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND > > 5868 nsivo 1 20 0 14456K 1016K zfs 0 0:00 0.00% ls > > > In ps: > USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND > > nsivo 5868 0.0 0.0 14456 1016 2- D+ 2:35PM 0:00.00 ls > > > Eventually the entire system hangs, and can't be shutdown cleanly. > > > What are the next steps to debug this? I'm a software developer, but am not familiar with kernel debugging. Is there a way to discover in which syscall ls is stuck? Ideally without requiring a crash dump? > > > Thanks for reading, > Nick > > > > -Nick > _______________________________________________ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org"