From owner-freebsd-stable@freebsd.org Sun Apr 9 17:52:54 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 77908D36BEC for ; Sun, 9 Apr 2017 17:52:54 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-pf0-x233.google.com (mail-pf0-x233.google.com [IPv6:2607:f8b0:400e:c00::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49726A9 for ; Sun, 9 Apr 2017 17:52:54 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-pf0-x233.google.com with SMTP id s16so23212244pfs.0 for ; Sun, 09 Apr 2017 10:52:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to; bh=kAsB2WbECwrXY/J8m0HoQmFe4E7EzJtWT4hUYvBSWEM=; b=t0byp6xTWSq/gDBLy45GBQDSQ1mGGEg93h++ma1P0CljB6r2VEs1MZknSsgRXyJeVQ BjctynkSVgEHxEysRjr9SoGllsDAO96zTF6H52Z7ELeAqTHQvPih4vbngLBC2Z4lsyj0 kxXIjCb8ZX+5AVLC7d9fP4IcMZcUTL8wOkPB2p98JW4p6f4Wfa3Z2pTgvP3b/qN7maCp oYyqikd8exLIZRi0kQ9owz2kFI8BlbbUlk5GKjFEwbidM9J8a97ch1rpknp7r7k7hT4w BJxUTIh7MrK4hvBHLUundr9rz7TRACLSq1R0VhbUqhVz8c9BTc7peew/ucgHLbKifqwo g3AA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to; bh=kAsB2WbECwrXY/J8m0HoQmFe4E7EzJtWT4hUYvBSWEM=; b=eA7tLVE9+cdkaj2c7ezxA/rbAN9Pz1Xl+hxTf+qaU2J8+dxfQbCm9jH0xYEDQeKB05 szzjE33rWwl4GQJpZLj83McJi5IFHCBHWRZre6+sWoVvJabwQJfpWMeVZjmtSpWkT5Gw Xnz/hUexNlaXjahBNLZ7DkX5/IWMjejz9NZaMAhiFK89N2+OEWSYzuvIguJE2WP1XDzv DNEMs68uGzSx08l0HFkE2/JAwPlE/J87O6EhXIYweP+KcNj65CuhhAf5Sp905G9gd889 e05IKJ7FSXHthWbfE1qyDYitmHw8rS+WksA/SqqynaHmjLWnqlJqIcaDwqVju0rRhU/j /F3g== X-Gm-Message-State: AFeK/H3okRt5gFec9cB+5Gw9W5Ddwy62ONVLU5eyEm8l2SDRFuUId/4z8XTnyj34Mdp0gmL6VnyYU63OyN8qXA== X-Received: by 10.84.212.130 with SMTP id e2mr64101296pli.140.1491760373704; Sun, 09 Apr 2017 10:52:53 -0700 (PDT) MIME-Version: 1.0 Sender: kob6558@gmail.com Received: by 10.100.182.97 with HTTP; Sun, 9 Apr 2017 10:52:53 -0700 (PDT) In-Reply-To: References: From: Kevin Oberman Date: Sun, 9 Apr 2017 10:52:53 -0700 X-Google-Sender-Auth: 3feZfp1VB-eErnMPkRa12TFQfpw Message-ID: Subject: Re: No USB? To: FreeBSD-STABLE Mailing List Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Apr 2017 17:52:54 -0000 On Sat, Apr 8, 2017 at 1:55 PM, Kevin Oberman wrote: > Today, for the first time in a couple of weeks, I plugged in a USB drive > to my 11-STABLE system (r316552). No device was created and usbconfig only > sees EHCI hubs: > ugen1.1: at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) > pwr=SAVE (0mA) > ugen0.1: at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) > pwr=SAVE (0mA) > > Seems like I should be seeing UHCI stuff, too. Even internal devices like > my webcam don't show up. > > I'm running a GENERIC kernel with the following exceptions: > nooptions SCHED_ULE # ULE scheduler > options SCHED_4BSD # 4BSD scheduler > options IEEE80211_DEBUG > > I tried updating my system and that made no difference. I booted up > Windows and it sees the USB drive just fine. > > Any things I should try or look at to try to figure out what is happening? > I really want to get an image of my system before moving in three days. > > This is looking more and more like a bug. I con't know why nobody else had seen it, but here is more information: Relevant limes from bot: ehci0: mem 0xf252a000-0xf252a3ff irq 16 at device 26.0 on pci0 usbus0: EHCI version 1.0 usbus0 on ehci0 ehci1: mem 0xf2529000-0xf25293ff irq 23 at device 29.0 on pci0 sbus1: EHCI version 1.0 usbus1 on ehci1 [...] usbus0: 480Mbps High Speed USB v2.0 usbus1: 480Mbps High Speed USB v2.0 ugen1.1: at usbus1 uhub0: on usbus1 ugen0.1: at usbus0 uhub1: on usbus0 uhub0: 3 ports with 3 removable, self powered uhub1: 3 ports with 3 removable, self powered usbus0: port reset timeout usbus1: port reset timeout uhub_reattach_port: port 1 reset failed, error=USB_ERR_TIMEOUT uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling port 1 uhub_reattach_port: port 1 reset failed, error=USB_ERR_TIMEOUT uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling port 1 usbconfig -d ugen1.1 reset produced: Apr 9 09:15:11 rogue kernel: uhub1: at usbus0, port 1, addr 1 (disconnected) Apr 9 09:15:11 rogue kernel: uhub1: Apr 9 09:15:11 rogue kernel: on usbus0 Apr 9 09:15:12 rogue kernel: uhub1: 3 ports with 3 removable, self powered Any ideas would be GREATLY appreciated as I can't backup or restore my system. I hope to boot a live version of 11-RELEASE if I can find one, and see if it works. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 From owner-freebsd-stable@freebsd.org Sun Apr 9 22:45:06 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3B3E0D36263 for ; Sun, 9 Apr 2017 22:45:06 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-pg0-x231.google.com (mail-pg0-x231.google.com [IPv6:2607:f8b0:400e:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0AEA3F07 for ; Sun, 9 Apr 2017 22:45:06 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-pg0-x231.google.com with SMTP id 21so96500228pgg.1 for ; Sun, 09 Apr 2017 15:45:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to; bh=4CavUudIoOqtadZq0rUFwm6kRjbcFpfhJMqT9Qb7BN8=; b=PBEG8XKLiB5oK5Jdo2eAMjMl9QwZzPSNTGbjclYuDfFpYkqAw0IMK6fXtDl/pt2Rir A61aGzS6QkRj9G3eV2Zb9e9Mht29B/ED29xL8r6Tgo0ZttQtntJT+D2At/u3e7RYye2c 0HP6RAMTeQ1ySlanxHcRnmcynSJZseMZPLUX+HSH+qDgeXkrIwuqOuZjroIiw6SFyZa1 nkWqKb6mo1lu8+6OTnEHpfjNtC6JuzX07MiTlTgSZSwd872a+rr7r8bRghHAamh11tfM kP0ub9NdV/+n0iyiyfBhsGDA9KZ7DC/krd5+c3bOXzTkmkd56VmiwHmDaKWEkRtmoLxu 5r0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to; bh=4CavUudIoOqtadZq0rUFwm6kRjbcFpfhJMqT9Qb7BN8=; b=ErJfDm72lCB9BjGBlFiWPE4y5SIGv4aYMresG15Z0KiPmAkHwvD4zTIOodrvGJStZW rkLbhasmImAyZRFvUveW3QZnTaZbncjws0eqwMItR0kU0flFw7IdNVhkQ+oc0oPoKhhJ k6kx4R7o2BdqWKUapR4jGpu9Mdu0StXrRcTRnPOwo0qPmt3gfat2HfH10A7+emz3plet A89eYLS34QB0n3BQCy6Ww/kEwJHl7xauAdOGnEWeEbbDBdk6hEiD5ixybCO6PcW+vt/Y LMjz3yXNPX0G8tFYJ/t4QBQHE6kHzsW5Tlfv/i5eK3Vjl47OI5LTb6NV492BbHMXBgiG Dh9A== X-Gm-Message-State: AFeK/H3z4c245Mnme7iG8F1gOf0YovjYNj0+RB5QtAt1SqVzgkgE3NIUaxL9SpNMHStN3S7A/DlYtKLfnZQ39A== X-Received: by 10.98.218.76 with SMTP id w12mr51675420pfl.162.1491777905502; Sun, 09 Apr 2017 15:45:05 -0700 (PDT) MIME-Version: 1.0 Sender: kob6558@gmail.com Received: by 10.100.182.97 with HTTP; Sun, 9 Apr 2017 15:45:05 -0700 (PDT) In-Reply-To: References: From: Kevin Oberman Date: Sun, 9 Apr 2017 15:45:05 -0700 X-Google-Sender-Auth: kUyR_9zDUeUleaJPfU_1TsZXr3s Message-ID: Subject: Re: No USB? To: FreeBSD-STABLE Mailing List , Hans Petter Selasky Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Apr 2017 22:45:06 -0000 I have opened https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=218513 on this issue. Bi-section in a problem as the update to LLVM a week ago breaks building old kernels. Hoping I can buildworld with the current compiler and then build the kernel. (Wonder if the new compiler could be the trigger for the problem I'm seeing?) Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 On Sun, Apr 9, 2017 at 10:52 AM, Kevin Oberman wrote: > On Sat, Apr 8, 2017 at 1:55 PM, Kevin Oberman wrote: > >> Today, for the first time in a couple of weeks, I plugged in a USB drive >> to my 11-STABLE system (r316552). No device was created and usbconfig only >> sees EHCI hubs: >> ugen1.1: at usbus1, cfg=0 md=HOST spd=HIGH >> (480Mbps) pwr=SAVE (0mA) >> ugen0.1: at usbus0, cfg=0 md=HOST spd=HIGH >> (480Mbps) pwr=SAVE (0mA) >> >> Seems like I should be seeing UHCI stuff, too. Even internal devices like >> my webcam don't show up. >> >> I'm running a GENERIC kernel with the following exceptions: >> nooptions SCHED_ULE # ULE scheduler >> options SCHED_4BSD # 4BSD scheduler >> options IEEE80211_DEBUG >> >> I tried updating my system and that made no difference. I booted up >> Windows and it sees the USB drive just fine. >> >> Any things I should try or look at to try to figure out what is >> happening? I really want to get an image of my system before moving in >> three days. >> >> This is looking more and more like a bug. I con't know why nobody else > had seen it, but here is more information: > Relevant limes from bot: > ehci0: mem 0xf252a000-0xf252a3ff irq > 16 at device 26.0 on pci0 > usbus0: EHCI version 1.0 > usbus0 on ehci0 > ehci1: mem 0xf2529000-0xf25293ff irq > 23 at device 29.0 on pci0 > sbus1: EHCI version 1.0 > usbus1 on ehci1 > [...] > usbus0: 480Mbps High Speed USB v2.0 > usbus1: 480Mbps High Speed USB v2.0 > ugen1.1: at usbus1 > uhub0: on usbus1 > ugen0.1: at usbus0 > uhub1: on usbus0 > uhub0: 3 ports with 3 removable, self powered > uhub1: 3 ports with 3 removable, self powered > usbus0: port reset timeout > usbus1: port reset timeout > uhub_reattach_port: port 1 reset failed, error=USB_ERR_TIMEOUT > uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling port 1 > uhub_reattach_port: port 1 reset failed, error=USB_ERR_TIMEOUT > uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling port 1 > > usbconfig -d ugen1.1 reset produced: > Apr 9 09:15:11 rogue kernel: uhub1: at usbus0, port 1, addr 1 > (disconnected) > Apr 9 09:15:11 rogue kernel: uhub1: > Apr 9 09:15:11 rogue kernel: 2.00/1.00, addr 1> on usbus0 > Apr 9 09:15:12 rogue kernel: uhub1: 3 ports with 3 removable, self powered > > Any ideas would be GREATLY appreciated as I can't backup or restore my > system. > > I hope to boot a live version of 11-RELEASE if I can find one, and see if > it works. > -- > Kevin Oberman, Part time kid herder and retired Network Engineer > E-mail: rkoberman@gmail.com > PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 > From owner-freebsd-stable@freebsd.org Mon Apr 10 04:42:47 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 97C83D36EE8 for ; Mon, 10 Apr 2017 04:42:47 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 833571738 for ; Mon, 10 Apr 2017 04:42:47 +0000 (UTC) (envelope-from julian@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 82485D36EE7; Mon, 10 Apr 2017 04:42:47 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8037BD36EE5 for ; Mon, 10 Apr 2017 04:42:47 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5FC7E1737 for ; Mon, 10 Apr 2017 04:42:46 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (106-68-206-144.dyn.iinet.net.au [106.68.206.144]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id v3A4gWpW047940 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 9 Apr 2017 21:42:37 -0700 (PDT) (envelope-from julian@freebsd.org) Subject: Re: moutnroot failing on zpools in Azure after upgrade from 10 to 11 due to lack of waiting for da0 To: Pete French , stable@freebsd.org References: <20170408110100.GB14604@brick> From: Julian Elischer Message-ID: <9f9bbb0e-2824-700f-1eac-8b904f91618b@freebsd.org> Date: Mon, 10 Apr 2017 12:42:26 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20170408110100.GB14604@brick> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2017 04:42:47 -0000 On 8/4/17 7:01 pm, Edward Tomasz Napierała wrote: > On 0313T1206, Pete French wrote: >> I have a number of machines in Azure, all booting from ZFS and, until >> the weekend, running 10.3 perfectly happily. >> >> I started upgrading these to 11. The first went fine, the second would >> not boot. Looking at the boot diagnistics it is having problems finding the >> root pool to mount. I see this is the diagnostic output: >> >> storvsc0: on vmbus0 >> Solaris: NOTICE: Cannot find the pool label for 'rpool' >> Mounting from zfs:rpool/ROOT/default failed with error 5. >> Root mount waiting for: storvsc >> (probe0:blkvsc0:0:storvsc1: 0:0): on vmbus0 >> storvsc scsi_status = 2 >> (da0:blkvsc0:0:0:0): UNMAPPED >> (probe1:blkvsc1:0:1:0): storvsc scsi_status = 2 >> hvheartbeat0: on vmbus0 >> da0 at blkvsc0 bus 0 scbus2 target 0 lun 0 >> >> As you can see, the drive da0 only appears after it has tried, and failed, >> to mount the root pool. > Does the same problem still happen with recent 11-STABLE? There is a fix for this floating around, we applied at work. Our systems are 10.3, but I think it wouldn't be a bad thing to add generally as it could (if we let it) solve the problem we sometimes see with nfs as well as with azure. p4 diff2 -du //depot/bugatti/FreeBSD-PZ/10.3/sys/kern/vfs_mountroot.c#1 //depot/bugatti/FreeBSD-PZ/10.3/sys/kern/vfs_mountroot.c#3 ==== //depot/bugatti/FreeBSD-PZ/10.3/sys/kern/vfs_mountroot.c#1 (text) - //depot/bugatti/FreeBSD-PZ/10.3/sys/kern/vfs_mountroot.c#3 (text) ==== content @@ -126,8 +126,8 @@ static int root_mount_mddev; static int root_mount_complete; -/* By default wait up to 3 seconds for devices to appear. */ -static int root_mount_timeout = 3; +/* By default wait up to 30 seconds for devices to appear. */ +static int root_mount_timeout = 30; TUNABLE_INT("vfs.mountroot.timeout", &root_mount_timeout); struct root_hold_token * @@ -690,7 +690,7 @@ char *errmsg; struct mntarg *ma; char *dev, *fs, *opts, *tok; - int delay, error, timeout; + int delay, error, timeout, err_stride; error = parse_token(conf, &tok); if (error) @@ -727,11 +727,20 @@ goto out; } + /* + * For ZFS we can't simply wait for a specific device + * as we only know the pool name. To work around this, + * parse_mount() will retry the mount later on. + * + * While retrying for NFS could be implemented similarly + * it is currently not supported. + */ + delay = hz / 10; + timeout = root_mount_timeout * hz; + if (strcmp(fs, "zfs") != 0 && strstr(fs, "nfs") == NULL && dev[0] != '\0' && !parse_mount_dev_present(dev)) { printf("mountroot: waiting for device %s ...\n", dev); - delay = hz / 10; - timeout = root_mount_timeout * hz; do { pause("rmdev", delay); timeout -= delay; @@ -741,16 +750,34 @@ goto out; } } + /* Timeout keeps counting down */ - ma = NULL; - ma = mount_arg(ma, "fstype", fs, -1); - ma = mount_arg(ma, "fspath", "/", -1); - ma = mount_arg(ma, "from", dev, -1); - ma = mount_arg(ma, "errmsg", errmsg, ERRMSGL); - ma = mount_arg(ma, "ro", NULL, 0); - ma = parse_mountroot_options(ma, opts); - error = kernel_mount(ma, MNT_ROOTFS); + err_stride=0; + do { + ma = NULL; + ma = mount_arg(ma, "fstype", fs, -1); + ma = mount_arg(ma, "fspath", "/", -1); + ma = mount_arg(ma, "from", dev, -1); + ma = mount_arg(ma, "errmsg", errmsg, ERRMSGL); + ma = mount_arg(ma, "ro", NULL, 0); + ma = parse_mountroot_options(ma, opts); + error = kernel_mount(ma, MNT_ROOTFS); + /* UFS only does it once */ + if (strcmp(fs, "zfs") != 0) + break; + timeout -= delay; + if (timeout > 0 && error) { + if (err_stride <= 0 ) { + printf("Mounting from %s:%s failed with error %d. " + "%d seconds left. Retrying.\n", fs, dev, error, + timeout / hz); + } + err_stride += 1; + err_stride %= 50; + pause("rmzfs", delay); + } + } while (timeout > 0 && error); out: if (error) { printf("Mounting from %s:%s failed with error %d", > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Mon Apr 10 17:56:42 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA464D379DE for ; Mon, 10 Apr 2017 17:56:42 +0000 (UTC) (envelope-from mahlon@martini.nu) Received: from martini.nu (martini.nu [69.163.39.203]) by mx1.freebsd.org (Postfix) with SMTP id C41C06DC for ; Mon, 10 Apr 2017 17:56:42 +0000 (UTC) (envelope-from mahlon@martini.nu) Received: (qmail 42959 invoked by uid 1000); 10 Apr 2017 17:50:03 -0000 Date: Mon, 10 Apr 2017 10:49:59 -0700 From: "Mahlon E. Smith" To: freebsd-stable@freebsd.org Subject: CAM timeouts at startup Message-ID: <20170410174959.GC73510@martini.nu> Mail-Followup-To: "Mahlon E. Smith" , freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="vGgW1X5XWziG23Ko" Content-Disposition: inline X-GPG-Fingerprint: 1F7E 39D8 BDEA E34B 1169 F285 0741 7A1E E630 75D2 X-GPG-Key: http://martini.nu/misc/key.txt X-Sysinfo: FreeBSD 10.3-RELEASE-p4 amd64 User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2017 17:56:42 -0000 --vGgW1X5XWziG23Ko Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi. Got some new machines running 11.0-RELEASE-p8 with a small pile of SSD= in them. Getting some strange CAM timeouts at boot, that dramatically delay startup times. These errors don't happen at all after boot, everything seems just fine. Any clues / tuning suggestions are appreciated. Relevant stuff from dmesg: mpr0: port 0x3000-0x30ff mem 0x92000000= -0x9200ffff,0x91f00000-0x91ffffff at device 0.0 numa-domain 0 on pci3 mpr0: IOCFacts : MsgVersion: 0x205 HeaderVersion: 0x2a00 IOCNumber: 0 IOCExceptions: 0x0 MaxChainDepth: 128 NumberOfPorts: 1 RequestCredit: 9680 ProductID: 0x2221 IOCRequestFrameSize: 32 MaxInitiators: 0 MaxTargets: 544 MaxSasExpanders: 125 MaxEnclosures: 126 HighPriorityCredit: 124 MaxReplyDescriptorPostQueueDepth: 65504 ReplyFrameSize: 32 MaxVolumes: 0 MaxDevHandle: 677 MaxPersistentEntries: 240 mpr0: Firmware: 13.00.00.00, Driver: 13.01.00.00-fbsd mpr0: IOCCapabilities: 7a85c da0 at mpr0 bus 0 scbus0 target 19 lun 0 da0: Fixed Direct Access SPC-4 SCSI device da0: Serial Number S2HTNX0H701448 =20 da0: 1200.000MB/s transfers da0: Command Queueing enabled da0: 915715MB (1875385008 512 byte sectors) da0: quirks=3D0x8<4K> mpr0: Sending reset from mprsas_send_abort for target ID 19 mpr0: Unfreezing devq for target ID 19 (da0:mpr0:0:19:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00=20 (da0:mpr0:0:19:0): CAM status: Command timeout (da0:mpr0:0:19:0): Retrying command (da0:mpr0:0:19:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00=20 (da0:mpr0:0:19:0): CAM status: SCSI Status Error (da0:mpr0:0:19:0): SCSI status: Check Condition (da0:mpr0:0:19:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, o= r bus device reset occurred) (da0:mpr0:0:19:0): Retrying command (per sense data) (noperiph:mpr0:0:4294967295:0): SMID 2 Aborting command 0xfffffe00026066= c0 mpr0: Sending reset from mprsas_send_abort for target ID 20 mpr0: Unfreezing devq for target ID 20 (da1:mpr0:0:20:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00=20 (da1:mpr0:0:20:0): CAM status: Command timeout (da1:mpr0:0:20:0): Retrying command (da1:mpr0:0:20:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00=20 (da1:mpr0:0:20:0): CAM status: SCSI Status Error (da1:mpr0:0:20:0): SCSI status: Check Condition (da1:mpr0:0:20:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, o= r bus device reset occurred) (da1:mpr0:0:20:0): Retrying command (per sense data) (noperiph:mpr0:0:4294967295:0): SMID 3 Aborting command 0xfffffe00026097= 50 mpr0: Sending reset from mprsas_send_abort for target ID 21 (da2:mpr0:0:21:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00=20 mpr0: (da2:mpr0:0:21:0): CAM status: Command timeout Unfreezing devq for target ID 21 (da2:mpr0:0:21:0): Retrying command (da2:mpr0:0:21:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00=20 (da2:mpr0:0:21:0): CAM status: SCSI Status Error (da2:mpr0:0:21:0): SCSI status: Check Condition (da2:mpr0:0:21:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, o= r bus device reset occurred) (da2:mpr0:0:21:0): Retrying command (per sense data) (noperiph:mpr0:0:4294967295:0): SMID 4 Aborting command 0xfffffe000260c7= e0 mpr0: Sending reset from mprsas_send_abort for target ID 22 mpr0: Unfreezing devq for target ID 22 (da3:mpr0:0:22:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00=20 (da3:mpr0:0:22:0): CAM status: Command timeout (da3:mpr0:0:22:0): Retrying command (da3:mpr0:0:22:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00=20 (da3:mpr0:0:22:0): CAM status: SCSI Status Error (da3:mpr0:0:22:0): SCSI status: Check Condition (da3:mpr0:0:22:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, o= r bus device reset occurred) (da3:mpr0:0:22:0): Retrying command (per sense data) (noperiph:mpr0:0:4294967295:0): SMID 5 Aborting command 0xfffffe0002614f= 10 mpr0: Sending reset from mprsas_send_abort for target ID 25 mpr0: Unfreezing devq for target ID 25 (da6:mpr0:0:25:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00=20 (da6:mpr0:0:25:0): CAM status: Command timeout (da6:mpr0:0:25:0): Retrying command (da6:mpr0:0:25:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00=20 (da6:mpr0:0:25:0): CAM status: SCSI Status Error (da6:mpr0:0:25:0): SCSI status: Check Condition (da6:mpr0:0:25:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, o= r bus device reset occurred) (da6:mpr0:0:25:0): Retrying command (per sense data) (noperiph:mpr0:0:4294967295:0): SMID 6 Aborting command 0xfffffe0002617f= a0 mpr0: Sending reset from mprsas_send_abort for target ID 26 mpr0: Unfreezing devq for target ID 26 (da7:mpr0:0:26:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00=20 (da7:mpr0:0:26:0): CAM status: Command timeout (da7:mpr0:0:26:0): Retrying command (da7:mpr0:0:26:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00=20 (da7:mpr0:0:26:0): CAM status: SCSI Status Error (da7:mpr0:0:26:0): SCSI status: Check Condition (da7:mpr0:0:26:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, o= r bus device reset occurred) (da7:mpr0:0:26:0): Retrying command (per sense data) -- Mahlon E. Smith =20 http://www.martini.nu/ --vGgW1X5XWziG23Ko Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJY68XDAAoJEAdBeh7mMHXS+XsQANjE4vyT/mv859vvhYqmiV5z +KCn3kpLtrjK7fMqgGa94y1JWVQe+d20ju7+mAtwXzHQBSu59McxfzLPfA8JspHx Obaq6rjdWL9npKwsueSAXerxyCHmjhTrF6rIv9CdtPbHhhF3se6craOWRgifkJa/ sZV5fYiND3gtiZTdIbL5XaZhGICBKMbARxARX10cfPAjTRMpU+djWOXSMoNl+jL8 FI1k9eKSTGnVGnj8dpUvvtSS0kKiEmHDTlM2QfTBPsEKOP+fOvL1GfEITGg7Gmi8 9qLqrLN3dr6DDZFLS0GB9GlM7+uI5NyEW3Vx1EAmK3eEMwvrTKRveb1RS247Cb0S jU5NJ7GJZh1A2i7LYwUDhzMIuZpMCfvxICVeizxc8ZARIWySccqF5/iEJyLPQ4ZN qMuHWEcVN4tQbeIuks34xRlNNqw3drfJyTbYtPYKqUhJ3I8V+9osVw5DsgVM9Z+4 kWL33WAqQCFNm33uPK0swba0ZrEfAAX18yL0VGo1nlkHfMr0yzHvXzJQaTCal6R5 YZr47GQ9BUj0PG3UEYzEGPfoEVBO3Y97uRpPCfvb6jlvtAZVMAQibS3z4V4qrlSR TnLKIyEtQN200wgQKyiCUD9czs5+SvDk3j2PYaDh+mR10OJc4nYiFRDI0p7wWIzG G8gUi8GrYu1WtuvE8kzE =iWBQ -----END PGP SIGNATURE----- --vGgW1X5XWziG23Ko-- From owner-freebsd-stable@freebsd.org Mon Apr 10 19:59:40 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 826F8D37E8B for ; Mon, 10 Apr 2017 19:59:40 +0000 (UTC) (envelope-from bsd-lists@bsdforge.com) Received: from udns.ultimatedns.net (static-24-113-41-81.wavecable.com [24.113.41.81]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5478B3B9 for ; Mon, 10 Apr 2017 19:59:39 +0000 (UTC) (envelope-from bsd-lists@bsdforge.com) Received: from ultimatedns.net (localhost [127.0.0.1]) by udns.ultimatedns.net (8.14.9/8.14.9) with ESMTP id v3AK14hr095989 for ; Mon, 10 Apr 2017 13:01:10 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) To: In-Reply-To: <20170410174959.GC73510@martini.nu> References: <20170410174959.GC73510@martini.nu> From: "Chris H" Subject: Re: CAM timeouts at startup Date: Mon, 10 Apr 2017 13:01:10 -0700 Content-Type: text/plain; charset=UTF-8; format=fixed MIME-Version: 1.0 Message-id: <24478d6acd3d88cb367c9f397f4f3bc1@ultimatedns.net> Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2017 19:59:40 -0000 On Mon, 10 Apr 2017 10:49:59 -0700 "Mahlon E. Smith" wrote > Hi. Got some new machines running 11.0-RELEASE-p8 with a small pile of SSD > in them. > > Getting some strange CAM timeouts at boot, that dramatically delay > startup times. These errors don't happen at all after boot, everything > seems just fine. > > Any clues / tuning suggestions are appreciated. > > > Relevant stuff from dmesg: > > mpr0: port 0x3000-0x30ff mem > 0x92000000-0x9200ffff,0x91f00000-0x91ffffff at device 0.0 numa-domain 0 on > pci3 mpr0: IOCFacts : > MsgVersion: 0x205 > HeaderVersion: 0x2a00 > IOCNumber: 0 > IOCExceptions: 0x0 > MaxChainDepth: 128 > NumberOfPorts: 1 > RequestCredit: 9680 > ProductID: 0x2221 > IOCRequestFrameSize: 32 > MaxInitiators: 0 > MaxTargets: 544 > MaxSasExpanders: 125 > MaxEnclosures: 126 > HighPriorityCredit: 124 > MaxReplyDescriptorPostQueueDepth: 65504 > ReplyFrameSize: 32 > MaxVolumes: 0 > MaxDevHandle: 677 > MaxPersistentEntries: 240 > mpr0: Firmware: 13.00.00.00, Driver: 13.01.00.00-fbsd > mpr0: IOCCapabilities: > 7a85c stDisc> > > da0 at mpr0 bus 0 scbus0 target 19 lun 0 > da0: Fixed Direct Access SPC-4 SCSI device > da0: Serial Number S2HTNX0H701448 > da0: 1200.000MB/s transfers > da0: Command Queueing enabled > da0: 915715MB (1875385008 512 byte sectors) > da0: quirks=0x8<4K> > > > mpr0: Sending reset from mprsas_send_abort for target ID 19 > mpr0: Unfreezing devq for target ID 19 > (da0:mpr0:0:19:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00 > (da0:mpr0:0:19:0): CAM status: Command timeout > (da0:mpr0:0:19:0): Retrying command > (da0:mpr0:0:19:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00 > (da0:mpr0:0:19:0): CAM status: SCSI Status Error > (da0:mpr0:0:19:0): SCSI status: Check Condition > (da0:mpr0:0:19:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) (da0:mpr0:0:19:0): Retrying command (per > sense data) (noperiph:mpr0:0:4294967295:0): SMID 2 Aborting > command 0xfffffe00026066c0 mpr0: Sending reset from mprsas_send_abort for > target ID 20 mpr0: Unfreezing devq for target ID 20 > (da1:mpr0:0:20:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00 > (da1:mpr0:0:20:0): CAM status: Command timeout > (da1:mpr0:0:20:0): Retrying command > (da1:mpr0:0:20:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00 > (da1:mpr0:0:20:0): CAM status: SCSI Status Error > (da1:mpr0:0:20:0): SCSI status: Check Condition > (da1:mpr0:0:20:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) (da1:mpr0:0:20:0): Retrying command (per > sense data) (noperiph:mpr0:0:4294967295:0): SMID 3 Aborting > command 0xfffffe0002609750 mpr0: Sending reset from mprsas_send_abort for > target ID 21 (da2:mpr0:0:21:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 > 00 mpr0: (da2:mpr0:0:21:0): CAM status: Command timeout > Unfreezing devq for target ID 21 > (da2:mpr0:0:21:0): Retrying command > (da2:mpr0:0:21:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00 > (da2:mpr0:0:21:0): CAM status: SCSI Status Error > (da2:mpr0:0:21:0): SCSI status: Check Condition > (da2:mpr0:0:21:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) (da2:mpr0:0:21:0): Retrying command (per > sense data) (noperiph:mpr0:0:4294967295:0): SMID 4 Aborting > command 0xfffffe000260c7e0 mpr0: Sending reset from mprsas_send_abort for > target ID 22 mpr0: Unfreezing devq for target ID 22 > (da3:mpr0:0:22:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00 > (da3:mpr0:0:22:0): CAM status: Command timeout > (da3:mpr0:0:22:0): Retrying command > (da3:mpr0:0:22:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00 > (da3:mpr0:0:22:0): CAM status: SCSI Status Error > (da3:mpr0:0:22:0): SCSI status: Check Condition > (da3:mpr0:0:22:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) (da3:mpr0:0:22:0): Retrying command (per > sense data) (noperiph:mpr0:0:4294967295:0): SMID 5 Aborting > command 0xfffffe0002614f10 mpr0: Sending reset from mprsas_send_abort for > target ID 25 mpr0: Unfreezing devq for target ID 25 > (da6:mpr0:0:25:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00 > (da6:mpr0:0:25:0): CAM status: Command timeout > (da6:mpr0:0:25:0): Retrying command > (da6:mpr0:0:25:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00 > (da6:mpr0:0:25:0): CAM status: SCSI Status Error > (da6:mpr0:0:25:0): SCSI status: Check Condition > (da6:mpr0:0:25:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) (da6:mpr0:0:25:0): Retrying command (per > sense data) (noperiph:mpr0:0:4294967295:0): SMID 6 Aborting > command 0xfffffe0002617fa0 mpr0: Sending reset from mprsas_send_abort for > target ID 26 mpr0: Unfreezing devq for target ID 26 > (da7:mpr0:0:26:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00 > (da7:mpr0:0:26:0): CAM status: Command timeout > (da7:mpr0:0:26:0): Retrying command > (da7:mpr0:0:26:0): READ(10). CDB: 28 00 6f c8 1a af 00 00 01 00 > (da7:mpr0:0:26:0): CAM status: SCSI Status Error > (da7:mpr0:0:26:0): SCSI status: Check Condition > (da7:mpr0:0:26:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) (da7:mpr0:0:26:0): Retrying command (per > sense data) > > > -- > Mahlon E. Smith In the past, when I've run into this issue. I add the following to loader.conf(5) kern.cam.boot_delay="" You'll want to tune it to find a "sweet spot". fe; on one of my boxes, it reads: kern.cam.boot_delay="7000" which is 7 seconds. HTH --Chris From owner-freebsd-stable@freebsd.org Mon Apr 10 22:39:13 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 704BED37528 for ; Mon, 10 Apr 2017 22:39:13 +0000 (UTC) (envelope-from mahlon@martini.nu) Received: from martini.nu (martini.nu [69.163.39.203]) by mx1.freebsd.org (Postfix) with SMTP id 58399992 for ; Mon, 10 Apr 2017 22:39:13 +0000 (UTC) (envelope-from mahlon@martini.nu) Received: (qmail 73032 invoked by uid 1000); 10 Apr 2017 22:39:15 -0000 Date: Mon, 10 Apr 2017 15:39:15 -0700 From: "Mahlon E. Smith" To: Chris H Cc: freebsd-stable@freebsd.org Subject: Re: CAM timeouts at startup Message-ID: <20170410223915.GD73510@martini.nu> Mail-Followup-To: "Mahlon E. Smith" , Chris H , freebsd-stable@freebsd.org References: <20170410174959.GC73510@martini.nu> <24478d6acd3d88cb367c9f397f4f3bc1@ultimatedns.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="HG+GLK89HZ1zG0kk" Content-Disposition: inline In-Reply-To: <24478d6acd3d88cb367c9f397f4f3bc1@ultimatedns.net> X-GPG-Fingerprint: 1F7E 39D8 BDEA E34B 1169 F285 0741 7A1E E630 75D2 X-GPG-Key: http://martini.nu/misc/key.txt X-Sysinfo: FreeBSD 10.3-RELEASE-p4 amd64 User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2017 22:39:13 -0000 --HG+GLK89HZ1zG0kk Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Apr 10, 2017, Chris H wrote: >=20 > In the past, when I've run into this issue. I add the following to > loader.conf(5) >=20 > kern.cam.boot_delay=3D"" > You'll want to tune it to find a "sweet spot". > fe; on one of my boxes, it reads: > kern.cam.boot_delay=3D"7000" > which is 7 seconds. Heya Chris, thanks for the reply. Trying it with the defaults from /boot/defaults/loader.conf doesn't seem to help so far, but I'll keep seeing if there's a sweet spot to discover. -- Mahlon E. Smith =20 http://www.martini.nu/ --HG+GLK89HZ1zG0kk Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJY7AmQAAoJEAdBeh7mMHXSPB4P/jc++1cHU1X8jW0gOWdZoitp gwNVrhMNRRRrFYijta91d00U10oRIKtX8OMcMPG3e4FooMocz7T8wC2zfuATGCLs tAVxJ3zilFBCWn1iWtQ7wJDMERIW53K5UdRNzyikfMJlIYdXv/U/nWxebzlULci5 hWWDs0xA6TGUyZ9dWNTRa4EhdxKD3sVVkALmEan3KdNATOEjOko3V5TKJzJ1wLPJ UJJcEDw2u3tNjZk7TftZPL7ENzDj0sLy4PuvS8Nx89hW5y2LS8giH9Pee5+ze8Lm Q0ie2J7mJZN9Th0ZHD/rVNOJM3bPLj8lIrTfnkBdQilhEUJKqhH0Zp2EFTuhwqX+ 9XMPI8lHXrpdIkYxTS+YqgMEGMfOIJmV1/7/N3CyBJ8BUSpef/1QzkSQluX70Evi corqVG4mfbFB4PTHYKiTgs20lkVpj6A59gV5zhM1+Kb+IPfQIqUW/ufTUX2Bfe+S qIOsohX8aP0lYNMqT74rEFxn86J9UVJdCP8qXdhVYklDoL3EhjQZsA0Oyy3OuuzE 9+S2Nvt5xOMaKIPYnNk/7WZerv54ekV3DXh2c34QXu4MHIYQ3aXx5F3O5934bWUt 0lwg1lojtbxuECoV0NikP4Em6rhrAcQLJigUiePK0C5DH/eQ8HYPwHFLvLPHONaJ O1Ttl5hZc3b+kRtZ9gdb =QO69 -----END PGP SIGNATURE----- --HG+GLK89HZ1zG0kk-- From owner-freebsd-stable@freebsd.org Tue Apr 11 15:43:17 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2EA7BD3978C for ; Tue, 11 Apr 2017 15:43:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1E3E8F1F for ; Tue, 11 Apr 2017 15:43:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v3BFhGuV061337 for ; Tue, 11 Apr 2017 15:43:16 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-stable@FreeBSD.org Subject: [Bug 213903] Kernel crashes from turnstile_broadcast (/usr/src/sys/kern/subr_turnstile.c:837) Date: Tue, 11 Apr 2017 15:43:17 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: crash X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: woodsb02@freebsd.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Apr 2017 15:43:17 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213903 --- Comment #29 from Ben Woods --- (In reply to Mateusz Guzik from comment #26) I have been running with the patch with no crashes. If you would like me to confirm whether the issue is still in CURRENT, I wi= ll have to upgrade to the latest CURRENT without the patch, and let you know i= f I get crashes again. Let me know if this would be helpful for you. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-stable@freebsd.org Wed Apr 12 18:01:12 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 04C78D3B2B5; Wed, 12 Apr 2017 18:01:12 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4529DB26; Wed, 12 Apr 2017 18:01:11 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [192.168.243.2] ([192.168.243.2]) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id v3CI15bD009360 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 12 Apr 2017 23:01:06 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1492020066; bh=W43TYZL3QHw6Gam1ePqA9lwtlPS106SGe3hjGS6htkk=; h=To:From:Subject:Date; b=cR1v64/5F+mQLyJaTjoThqRJlISXGcsZQe0Y8Er9tk2DKSftnlhoiw8o63dUDhiz6 MN3iMvpLveI1hsQ7LarbfFr12LYQOmeYzA8cR2c9bJXLjKZ7zSfQx8NA1JYWxZ4rgP fBP0bd8wQwKM2NhtP02bFSQnGDjVr7mm1F8uA4vM= To: freebsd-stable@FreeBSD.org, FreeBSD FS From: "Eugene M. Zheganin" Subject: zpool list show nonsense on raidz pools, at least it looks like it for me Message-ID: <71ef8400-94ec-1f59-3b2b-bb576ad65b89@norma.perm.ru> Date: Wed, 12 Apr 2017 23:01:10 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Apr 2017 18:01:12 -0000 Hi, It's not my first letter where I fail to understand the space usage from zfs utilities, and in previous ones I was kind of convinced that I just read it wrong, but not this time I guess. See for yourself: [emz@san01:~]> zpool list data NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT data 17,4T 7,72T 9,66T - 46% 44% 1.00x ONLINE - Here' as I understand it, zpool says that less than a half of the pool is used. As far as I know this is very complicated when it comes to the radiz pools. Let's see: [emz@san01:~]> zfs list -t all data NAME USED AVAIL REFER MOUNTPOINT data 13,3T 186G 27,2K /data So, if we won't investigate further, it looks like that only 186G is free. Spoiling - this is the real free space amount, because I've just managed to free 160 gigs of data, and I really know I was short on space when sending 30 Gb dataset, because zfs was saying "Not enough free space". So, let's investigate further: [emz@san01:~]> zfs list -t all | more NAME USED AVAIL REFER MOUNTPOINT data 13,3T 186G 27,2K /data data/esx 5,23T 186G 27,2K /data/esx data/esx/boot-esx01 8,25G 193G 561M - data/esx/boot-esx02 8,25G 193G 561M - data/esx/boot-esx03 8,25G 193G 561M - data/esx/boot-esx04 8,25G 193G 561M - data/esx/boot-esx05 8,25G 193G 561M - data/esx/boot-esx06 8,25G 193G 561M - data/esx/boot-esx07 8,25G 193G 962M - data/esx/boot-esx08 8,25G 193G 562M - data/esx/boot-esx09 8,25G 193G 562M - data/esx/boot-esx10 8,25G 193G 595M - data/esx/boot-esx11 8,25G 193G 539M - data/esx/boot-esx12 8,25G 193G 539M - data/esx/boot-esx13 8,25G 193G 539M - data/esx/boot-esx14 8,25G 193G 539M - data/esx/boot-esx15 8,25G 193G 539M - data/esx/boot-esx16 8,25G 193G 541M - data/esx/boot-esx17 8,25G 193G 540M - data/esx/boot-esx18 8,25G 193G 539M - data/esx/boot-esx19 8,25G 193G 542M - data/esx/boot-esx20 8,25G 194G 12,8K - data/esx/boot-esx21 8,25G 194G 12,8K - data/esx/boot-esx22 8,25G 193G 913M - data/esx/boot-esx23 8,25G 193G 558M - data/esx/boot-esx24 8,25G 194G 12,8K - data/esx/boot-esx25 8,25G 194G 12,8K - data/esx/boot-esx26 8,25G 194G 12,8K - data/esx/shared 5,02T 2,59T 2,61T - data/reference 6,74T 4,17T 2,73T - data/reference@ver7_214 127M - 2,73T - data/reference@ver2_739 12,8M - 2,73T - data/reference@ver2_740 5,80M - 2,73T - data/reference@ver2_741 4,55M - 2,73T - data/reference@ver2_742 993K - 2,73T - data/reference-ver2_739-worker100 1,64G 186G 2,73T - data/reference-ver2_739-worker101 254M 186G 2,73T - data/reference-ver2_739-worker102 566K 186G 2,73T - data/reference-ver2_739-worker103 260M 186G 2,73T - data/reference-ver2_739-worker104 8,74G 186G 2,73T - data/reference-ver2_739-worker105 4,19G 186G 2,73T - data/reference-ver2_739-worker106 1,72G 186G 2,73T - data/reference-ver2_739-worker107 282M 186G 2,73T - data/reference-ver2_739-worker108 1,27M 186G 2,73T - data/reference-ver2_739-worker109 8,74G 186G 2,73T - data/reference-ver2_739-worker110 8,74G 186G 2,73T - data/reference-ver2_739-worker111 8,74G 186G 2,73T - data/reference-ver2_739-worker112 8,74G 186G 2,73T - data/reference-ver2_739-worker113 838K 186G 2,73T - data/reference-ver2_739-worker114 8,74G 186G 2,73T - data/reference-ver2_739-worker115 8,74G 186G 2,73T - data/reference-ver2_739-worker116 8,74G 186G 2,73T - data/reference-ver2_739-worker117 8,74G 186G 2,73T - data/reference-ver2_739-worker118 8,74G 186G 2,73T - data/reference-ver2_739-worker119 8,74G 186G 2,73T - data/reference-ver2_739-worker120 8,74G 186G 2,73T - data/reference-ver2_739-worker121 8,74G 186G 2,73T - data/reference-ver2_739-worker122 8,74G 186G 2,73T - data/reference-ver2_739-worker123 8,74G 186G 2,73T - data/reference-ver2_739-worker124 8,74G 186G 2,73T - data/reference-ver2_739-worker125 8,74G 186G 2,73T - data/reference-ver2_739-worker126 8,74G 186G 2,73T - data/reference-ver2_739-worker127 8,74G 186G 2,73T - data/reference-ver2_739-worker128 8,74G 186G 2,73T - data/reference-ver2_739-worker129 8,74G 186G 2,73T - data/reference-ver2_739-worker130 8,74G 186G 2,73T - data/reference-ver2_739-worker131 8,74G 186G 2,73T - data/reference-ver2_739-worker132 8,74G 186G 2,73T - data/reference-ver2_739-worker133 8,74G 186G 2,73T - data/reference-ver2_739-worker134 8,74G 186G 2,73T - data/reference-ver2_739-worker135 8,74G 186G 2,73T - data/reference-ver2_739-worker136 8,74G 186G 2,73T - data/reference-ver2_739-worker137 8,74G 186G 2,73T - data/reference-ver2_739-worker138 8,74G 186G 2,73T - data/reference-ver2_739-worker139 8,74G 186G 2,73T - data/reference-ver2_739-worker140 8,74G 186G 2,73T - data/reference-ver2_739-worker141 8,74G 186G 2,73T - data/reference-ver2_739-worker142 8,74G 186G 2,73T - data/reference-ver2_739-worker143 8,73G 186G 2,73T - data/reference-ver2_739-worker144 9,16G 186G 2,73T - data/reference-ver2_739-worker145 8,74G 186G 2,73T - data/reference-ver2_739-worker146 8,74G 186G 2,73T - data/reference-ver2_739-worker147 8,74G 186G 2,73T - data/reference-ver2_739-worker148 8,74G 186G 2,73T - data/reference-ver2_739-worker149 8,74G 186G 2,73T - data/reference-ver2_739-worker150 8,74G 186G 2,73T - data/reference-ver2_739-worker151 8,74G 186G 2,73T - data/reference-ver2_739-worker152 8,74G 186G 2,73T - data/reference-ver2_739-worker53 385K 186G 2,73T - data/reference-ver2_739-worker58 9,49M 186G 2,73T - data/reference-ver2_739-worker81 8,20G 186G 2,73T - data/reference-ver2_739-worker82 7,64G 186G 2,73T - data/reference-ver2_739-worker83 6,84G 186G 2,73T - data/reference-ver2_739-worker84 5,95G 186G 2,73T - data/reference-ver2_739-worker85 5,09G 186G 2,73T - data/reference-ver2_739-worker86 3,76G 186G 2,73T - data/reference-ver2_739-worker87 4,06G 186G 2,73T - data/reference-ver2_739-worker88 1,92G 186G 2,73T - data/reference-ver2_739-worker89 6,83G 186G 2,73T - data/reference-ver2_739-worker90 6,55G 186G 2,73T - data/reference-ver2_739-worker91 5,25G 186G 2,73T - data/reference-ver2_739-worker92 4,18G 186G 2,73T - data/reference-ver2_739-worker93 3,18G 186G 2,73T - data/reference-ver2_739-worker94 983M 186G 2,73T - data/reference-ver2_739-worker95 525M 186G 2,73T - data/reference-ver2_739-worker96 650M 186G 2,73T - data/reference-ver2_739-worker97 5,78G 186G 2,73T - data/reference-ver2_739-worker98 4,81G 186G 2,73T - data/reference-ver2_739-worker99 4,93G 186G 2,73T - data/reference-ver2_740-worker56 361K 186G 2,73T - data/reference-ver2_741-worker74 3,98M 186G 2,73T - data/reference-ver2_742-worker02 934K 186G 2,73T - data/reference-ver2_742-worker03 935K 186G 2,73T - data/reference-ver2_742-worker04 935K 186G 2,73T - data/reference-ver2_742-worker05 935K 186G 2,73T - data/reference-ver2_742-worker06 935K 186G 2,73T - data/reference-ver2_742-worker08 934K 186G 2,73T - data/reference-ver2_742-worker09 15,9M 186G 2,73T - data/reference-ver2_742-worker10 16,2M 186G 2,73T - data/reference-ver2_742-worker11 15,4M 186G 2,73T - data/reference-ver2_742-worker12 935K 186G 2,73T - data/reference-ver2_742-worker13 15,9M 186G 2,73T - data/reference-ver2_742-worker14 935K 186G 2,73T - data/reference-ver2_742-worker15 15,5M 186G 2,73T - data/reference-ver2_742-worker16 934K 186G 2,73T - data/reference-ver2_742-worker17 934K 186G 2,73T - data/reference-ver2_742-worker18 935K 186G 2,73T - data/reference-ver2_742-worker19 531M 186G 2,73T - data/reference-ver2_742-worker20 935K 186G 2,73T - data/reference-ver2_742-worker21 935K 186G 2,73T - data/reference-ver2_742-worker22 5,27G 186G 2,73T - data/reference-ver2_742-worker23 934K 186G 2,73T - data/reference-ver2_742-worker24 5,27G 186G 2,73T - data/reference-ver2_742-worker25 934K 186G 2,73T - data/reference-ver2_742-worker26 934K 186G 2,73T - data/reference-ver2_742-worker27 15,8M 186G 2,73T - data/reference-ver2_742-worker28 935K 186G 2,73T - data/reference-ver2_742-worker29 934K 186G 2,73T - data/reference-ver2_742-worker30 934K 186G 2,73T - data/reference-ver2_742-worker31 934K 186G 2,73T - data/reference-ver2_742-worker32 444M 186G 2,73T - data/reference-ver2_742-worker33 935K 186G 2,73T - data/reference-ver2_742-worker34 15,9M 186G 2,73T - data/reference-ver2_742-worker35 1,12M 186G 2,73T - data/reference-ver2_742-worker36 935K 186G 2,73T - data/reference-ver2_742-worker37 5,27G 186G 2,73T - data/reference-ver2_742-worker38 935K 186G 2,73T - data/reference-ver2_742-worker39 935K 186G 2,73T - data/reference-ver2_742-worker40 935K 186G 2,73T - data/reference-ver2_742-worker41 15,5M 186G 2,73T - data/reference-ver2_742-worker42 934K 186G 2,73T - data/reference-ver2_742-worker43 15,4M 186G 2,73T - data/reference-ver2_742-worker44 429M 186G 2,73T - data/reference-ver2_742-worker45 15,5M 186G 2,73T - data/reference-ver2_742-worker46 935K 186G 2,73T - data/reference-ver2_742-worker47 934K 186G 2,73T - data/reference-ver2_742-worker48 15,5M 186G 2,73T - data/reference-ver2_742-worker49 935K 186G 2,73T - data/reference-ver2_742-worker50 934K 186G 2,73T - data/reference-ver2_742-worker51 935K 186G 2,73T - data/reference-ver2_742-worker52 935K 186G 2,73T - data/reference-ver2_742-worker54 935K 186G 2,73T - data/reference-ver2_742-worker55 934K 186G 2,73T - data/reference-ver2_742-worker57 935K 186G 2,73T - data/reference-ver2_742-worker59 934K 186G 2,73T - data/reference-ver2_742-worker60 15,5M 186G 2,73T - data/reference-ver2_742-worker61 934K 186G 2,73T - data/reference-ver2_742-worker62 934K 186G 2,73T - data/reference-ver2_742-worker63 934K 186G 2,73T - data/reference-ver2_742-worker64 15,5M 186G 2,73T - data/reference-ver2_742-worker65 934K 186G 2,73T - data/reference-ver2_742-worker66 935K 186G 2,73T - data/reference-ver2_742-worker67 934K 186G 2,73T - data/reference-ver2_742-worker68 935K 186G 2,73T - data/reference-ver2_742-worker69 934K 186G 2,73T - data/reference-ver2_742-worker70 935K 186G 2,73T - data/reference-ver2_742-worker71 15,5M 186G 2,73T - data/reference-ver2_742-worker72 935K 186G 2,73T - data/reference-ver2_742-worker73 935K 186G 2,73T - data/reference-ver2_742-worker75 934K 186G 2,73T - data/reference-ver2_742-worker76 935K 186G 2,73T - data/reference-ver2_742-worker77 934K 186G 2,73T - data/reference-ver2_742-worker78 15,7M 186G 2,73T - data/reference-ver2_742-worker79 15,4M 186G 2,73T - data/reference-ver2_742-worker80 935K 186G 2,73T - data/reference-ver7_214-worker07 2,19M 186G 2,73T - data/reference-ver7_214-worker53 2,18M 186G 2,73T - data/userdata 825G 186G 27,2K /data/userdata data/userdata/userdata 27,2K 186G 27,2K /data/userdata/userdata data/userdata/worker01 8,25G 190G 3,77G - data/userdata/worker02 8,25G 190G 3,48G - data/userdata/worker03 8,25G 190G 4,07G - data/userdata/worker04 8,25G 191G 2,99G - data/userdata/worker05 8,25G 190G 3,46G - data/userdata/worker06 8,25G 191G 2,89G - data/userdata/worker07 8,25G 190G 3,65G - data/userdata/worker08 8,25G 190G 3,55G - data/userdata/worker09 8,25G 190G 3,64G - data/userdata/worker10 8,25G 190G 3,85G - data/userdata/worker11 8,25G 191G 3,28G - data/userdata/worker12 8,25G 191G 3,19G - data/userdata/worker13 8,25G 191G 2,93G - data/userdata/worker14 8,25G 191G 2,89G - data/userdata/worker15 8,25G 190G 3,57G - data/userdata/worker153 8,25G 194G 12,8K - data/userdata/worker154 8,25G 194G 12,8K - data/userdata/worker155 8,25G 194G 12,8K - data/userdata/worker156 8,25G 194G 12,8K - data/userdata/worker157 8,25G 194G 12,8K - data/userdata/worker158 8,25G 194G 12,8K - data/userdata/worker159 8,25G 194G 12,8K - data/userdata/worker16 8,25G 190G 4,23G - data/userdata/worker160 8,25G 194G 12,8K - data/userdata/worker161 8,25G 194G 12,8K - data/userdata/worker162 8,25G 194G 12,8K - data/userdata/worker163 8,25G 194G 12,8K - data/userdata/worker164 8,25G 194G 12,8K - data/userdata/worker165 8,25G 194G 12,8K - data/userdata/worker166 8,25G 194G 12,8K - data/userdata/worker167 8,25G 194G 12,8K - data/userdata/worker168 8,25G 194G 12,8K - data/userdata/worker169 8,25G 194G 12,8K - data/userdata/worker17 8,25G 190G 3,63G - data/userdata/worker170 8,25G 194G 12,8K - data/userdata/worker171 8,25G 194G 12,8K - data/userdata/worker172 8,25G 194G 12,8K - data/userdata/worker18 8,25G 191G 3,19G - data/userdata/worker19 8,25G 191G 3,02G - data/userdata/worker20 8,25G 192G 2,35G - data/userdata/worker21 8,25G 188G 5,42G - data/userdata/worker22 8,25G 191G 2,76G - data/userdata/worker23 8,25G 191G 2,61G - data/userdata/worker24 8,25G 191G 3,22G - data/userdata/worker25 8,25G 189G 4,97G - data/userdata/worker26 8,25G 191G 3,23G - data/userdata/worker27 8,25G 189G 4,90G - data/userdata/worker28 8,25G 189G 5,31G - data/userdata/worker29 8,25G 189G 4,67G - data/userdata/worker30 8,25G 190G 3,46G - data/userdata/worker31 8,25G 191G 3,36G - data/userdata/worker32 8,25G 190G 4,11G - data/userdata/worker33 8,25G 190G 4,31G - data/userdata/worker34 8,25G 191G 3,31G - data/userdata/worker35 8,25G 190G 4,23G - data/userdata/worker36 8,25G 191G 3,24G - data/userdata/worker37 8,25G 191G 3,16G - data/userdata/worker38 8,25G 191G 2,89G - data/userdata/worker39 8,25G 189G 4,49G - data/userdata/worker40 8,25G 190G 4,21G - data/userdata/worker41 8,25G 190G 3,42G - data/userdata/worker42 8,25G 191G 3,32G - data/userdata/worker43 8,25G 190G 3,71G - data/userdata/worker44 8,25G 191G 3,33G - data/userdata/worker45 8,25G 189G 5,04G - data/userdata/worker46 8,25G 190G 3,76G - data/userdata/worker47 8,25G 189G 5,13G - data/userdata/worker48 8,25G 190G 3,81G - data/userdata/worker49 8,25G 191G 2,72G - data/userdata/worker50 8,25G 190G 3,78G - data/userdata/worker51 8,25G 191G 3,08G - data/userdata/worker52 8,25G 190G 3,38G - data/userdata/worker53 8,25G 190G 4,10G - data/userdata/worker54 8,25G 190G 3,48G - data/userdata/worker55 8,25G 189G 4,39G - data/userdata/worker56 8,25G 190G 3,58G - data/userdata/worker57 8,25G 190G 3,97G - data/userdata/worker58 8,25G 190G 4,02G - data/userdata/worker59 8,25G 190G 3,57G - data/userdata/worker60 8,25G 190G 4,04G - data/userdata/worker61 8,25G 190G 3,73G - data/userdata/worker62 8,25G 191G 3,21G - data/userdata/worker63 8,25G 191G 3,37G - data/userdata/worker64 8,25G 189G 4,77G - data/userdata/worker65 8,25G 189G 5,34G - data/userdata/worker66 8,25G 190G 3,50G - data/userdata/worker67 8,25G 190G 3,84G - data/userdata/worker68 8,25G 189G 4,73G - data/userdata/worker69 8,25G 191G 3,20G - data/userdata/worker70 8,25G 190G 3,96G - data/userdata/worker71 8,25G 189G 5,11G - data/userdata/worker72 8,25G 191G 2,92G - data/userdata/worker73 8,25G 189G 4,53G - data/userdata/worker74 8,25G 189G 4,39G - data/userdata/worker75 8,25G 190G 4,26G - data/userdata/worker76 8,25G 189G 4,67G - data/userdata/worker77 8,25G 189G 5,30G - data/userdata/worker78 8,25G 190G 3,98G - data/userdata/worker79 8,25G 190G 4,10G - data/userdata/worker80 8,25G 190G 3,48G - This are getting really complicated now. What I don't understand is: - why the amount of free space changes from dataset to dataset ? I mean they all share the same free space pool, all have the same refreservation=none, but the AVAIL differs. When it comes to workerX datasets, it differs slightly, but when it comes to the large zvols, like esx/shared or reference, it differs a lot ! - why the esx/shared and reference datasets are shown like they can be enlarged ? I mean, I really don't have THAT much of free space. Here are their properties: [emz@san01:~]> zfs get all data/esx/shared NAME PROPERTY VALUE SOURCE data/esx/shared type volume - data/esx/shared creation вт февр. 21 10:19 2017 - data/esx/shared used 5,02T - data/esx/shared available 2,59T - data/esx/shared referenced 2,61T - data/esx/shared compressratio 1.81x - data/esx/shared reservation none default data/esx/shared volsize 5T local data/esx/shared volblocksize 64K - data/esx/shared checksum on default data/esx/shared compression lzjb inherited from data data/esx/shared readonly off default data/esx/shared copies 1 default data/esx/shared refreservation 5,02T local data/esx/shared primarycache all default data/esx/shared secondarycache all default data/esx/shared usedbysnapshots 0 - data/esx/shared usedbydataset 2,61T - data/esx/shared usedbychildren 0 - data/esx/shared usedbyrefreservation 2,41T - data/esx/shared logbias latency default data/esx/shared dedup off default data/esx/shared mlslabel - data/esx/shared sync standard default data/esx/shared refcompressratio 1.81x - data/esx/shared written 2,61T - data/esx/shared logicalused 4,71T - data/esx/shared logicalreferenced 4,71T - data/esx/shared volmode default default data/esx/shared snapshot_limit none default data/esx/shared snapshot_count none default data/esx/shared redundant_metadata all default [emz@san01:~]> zfs get all data/reference NAME PROPERTY VALUE SOURCE data/reference type volume - data/reference creation вт февр. 21 11:12 2017 - data/reference used 6,74T - data/reference available 4,17T - data/reference referenced 2,73T - data/reference compressratio 1.08x - data/reference reservation none default data/reference volsize 3,97T local data/reference volblocksize 64K - data/reference checksum on default data/reference compression lzjb local data/reference readonly off default data/reference copies 1 default data/reference refreservation 3,98T local data/reference primarycache all default data/reference secondarycache all default data/reference usedbysnapshots 21,6G - data/reference usedbydataset 2,73T - data/reference usedbychildren 0 - data/reference usedbyrefreservation 3,98T - data/reference logbias latency default data/reference dedup off default data/reference mlslabel - data/reference sync standard default data/reference refcompressratio 1.08x - data/reference written 1,10M - data/reference logicalused 2,98T - data/reference logicalreferenced 2,95T - data/reference volmode default default data/reference snapshot_limit none default data/reference snapshot_count none default data/reference redundant_metadata all default Could please someone explain why they show as having like half of the total pool space as AVAIL ? I thing this is directly related to the fact that zpool list shows only 44% of the total pool space is used. And I use this value to monitor the pool space usage, looks like I'm totally failing with this. I also don't understand whe the zvol of the size 3.97T really uses 6.74T of the space. I found an article, explaing that the volblocksize and the sector size has to do something with this, and this happens when the device block size is 4k, and volblocksize is default, thus 8k. Mine disks sector size is 512 native, so this is really not the case. I'm also having equal number of disks in vdevs, and they are 5: [root@san01:~]# zpool status data pool: data state: ONLINE scan: scrub repaired 111K in 40h42m with 0 errors on Wed Mar 1 03:34:56 2017 config: NAME STATE READ WRITE CKSUM data ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 da0 ONLINE 0 0 0 da1 ONLINE 0 0 0 diskid/DISK-96JS100XTB4V ONLINE 0 0 0 diskid/DISK-96JS101TTB4V ONLINE 0 0 0 diskid/DISK-96JS100MTB4V ONLINE 0 0 0 raidz1-1 ONLINE 0 0 0 diskid/DISK-96JS1016TB4V ONLINE 0 0 0 diskid/DISK-96JS1002TB4V ONLINE 0 0 0 diskid/DISK-96LS1066TB4V ONLINE 0 0 0 diskid/DISK-96JS101JTB4V ONLINE 0 0 0 diskid/DISK-96LS106TTB4V ONLINE 0 0 0 errors: No known data errors So I guess poor vdev alignment is also not the case. I used to resize the zvol a couple of times, and I suspect this is the main reason, because I've read in another article that this is probably the case when "poor alignment" happens. I will really appreciate if someone will explain this. Thanks. From owner-freebsd-stable@freebsd.org Wed Apr 12 19:20:39 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 18472D3B3C1; Wed, 12 Apr 2017 19:20:39 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-yw0-x229.google.com (mail-yw0-x229.google.com [IPv6:2607:f8b0:4002:c05::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CC4ECCC2; Wed, 12 Apr 2017 19:20:38 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-yw0-x229.google.com with SMTP id j9so16399950ywj.3; Wed, 12 Apr 2017 12:20:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=k/kotrJScR2Ps5zMciC8N6nuMXXQnEkykIYvsdVifk0=; b=EilNL9aXFQ1sGHpxzyQjaNlyL0CYn6RKRMowUz7yyDQdsrkimjHOpfWb8Dgt0gwLxT PbNSfyh5EyQt06OLLl8Jg67e8CoifoBQS8D9axd4aN2rWvxrhfZB6Q6cPEimeUSLBcr1 4vsK17vPmmxbeR9k7vPwXqCJ3CAdiE4y+WS5fJU/5zyNTiJ0wOmYaYaEtcDnxjm2eXZH N5cUTr1NyCPTIewiSeQgaVj+b0k8UIxydNl0hAgGLP9z15cFr87PvVav7rje8FAaNP9B V9OJyQ4mgUHsZiNyQWuLgvKWx5IMUVqPCERoT/0VIcica2ihJpFuETi2FsXgkRauX0ih /mmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=k/kotrJScR2Ps5zMciC8N6nuMXXQnEkykIYvsdVifk0=; b=ah088oGgmexz6Q9lnskUplFlJwWrcrH0gMpo+negFNOuFuiqYS0w5iifTf1+DIoPm2 dllnJ/5YiFvSofXHX1lWj1hHQuB/65faVzE3Mtz3s5apACjpAMblvlZ5M1vNDIdM8WVb AfSJmkgERN86hWYPisima1nE3LNH16BhdU4hujcuKIVQtf8MofYL1GgAd7e0FZ+bTSiH C/OJ+PXxmZs9wiIW98nc9weED3jesssljKW2jng70bYcDS4BzCzMHgs9MajEPUWfda4W Zm2ABR6oumVWK6/0deaINhUZl4FXsvZN9KLseMROy/PZDtREk6396LZImJuv72PCMCl5 qEiw== X-Gm-Message-State: AN3rC/719+HO1sA+/HLH3bddDCtv2zfZ76DypOrJcVqnfI3EcNlkzRWvGdS1/rwrsPX7uu3cTgA3OcAnUBz1ig== X-Received: by 10.129.173.69 with SMTP id l5mr8155955ywk.351.1492024837733; Wed, 12 Apr 2017 12:20:37 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.129.20.214 with HTTP; Wed, 12 Apr 2017 12:20:37 -0700 (PDT) In-Reply-To: <71ef8400-94ec-1f59-3b2b-bb576ad65b89@norma.perm.ru> References: <71ef8400-94ec-1f59-3b2b-bb576ad65b89@norma.perm.ru> From: Alan Somers Date: Wed, 12 Apr 2017 13:20:37 -0600 X-Google-Sender-Auth: jbmXM0ZpdrCEXnabGKrRoNu6EjE Message-ID: Subject: Re: zpool list show nonsense on raidz pools, at least it looks like it for me To: "Eugene M. Zheganin" Cc: FreeBSD , FreeBSD FS Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Apr 2017 19:20:39 -0000 On Wed, Apr 12, 2017 at 12:01 PM, Eugene M. Zheganin wrote: > Hi, > > > It's not my first letter where I fail to understand the space usage from zfs > utilities, and in previous ones I was kind of convinced that I just read it > wrong, but not this time I guess. See for yourself: > > > [emz@san01:~]> zpool list data > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > data 17,4T 7,72T 9,66T - 46% 44% 1.00x ONLINE - > > > Here' as I understand it, zpool says that less than a half of the pool is > used. As far as I know this is very complicated when it comes to the radiz > pools. Let's see: > > > [emz@san01:~]> zfs list -t all data > NAME USED AVAIL REFER MOUNTPOINT > data 13,3T 186G 27,2K /data > > > So, if we won't investigate further, it looks like that only 186G is free. > Spoiling - this is the real free space amount, because I've just managed to > free 160 gigs of data, and I really know I was short on space when sending > 30 Gb dataset, because zfs was saying "Not enough free space". So, let's > investigate further: > > > [emz@san01:~]> zfs list -t all | more > NAME USED AVAIL REFER MOUNTPOINT > data 13,3T 186G 27,2K /data > data/esx 5,23T 186G 27,2K /data/esx ... > data/esx/boot-esx26 8,25G 194G 12,8K - > data/esx/shared 5,02T 2,59T 2,61T - > data/reference 6,74T 4,17T 2,73T - > data/reference@ver7_214 127M - 2,73T - > data/reference@ver2_739 12,8M - 2,73T - > data/reference@ver2_740 5,80M - 2,73T - > data/reference@ver2_741 4,55M - 2,73T - > data/reference@ver2_742 993K - 2,73T - > data/reference-ver2_739-worker100 1,64G 186G 2,73T - ... > > > This are getting really complicated now. > > What I don't understand is: > > - why the amount of free space changes from dataset to dataset ? I mean they > all share the same free space pool, all have the same refreservation=none, > but the AVAIL differs. When it comes to workerX datasets, it differs > slightly, but when it comes to the large zvols, like esx/shared or > reference, it differs a lot ! > > - why the esx/shared and reference datasets are shown like they can be > enlarged ? I mean, I really don't have THAT much of free space. > > > Here are their properties: > > > [emz@san01:~]> zfs get all data/esx/shared > NAME PROPERTY VALUE SOURCE ... > data/esx/shared refreservation 5,02T local ... > [emz@san01:~]> zfs get all data/reference > NAME PROPERTY VALUE SOURCE ... > data/reference refreservation 3,98T local ... > > > Could please someone explain why they show as having like half of the total > pool space as AVAIL ? I thing this is directly related to the fact that > zpool list shows only 44% of the total pool space is used. And I use this > value to monitor the pool space usage, looks like I'm totally failing with > this. > Some of your datasets have refreservations. That's why. > > I also don't understand whe the zvol of the size 3.97T really uses 6.74T of > the space. I found an article, explaing that the volblocksize and the sector > size has to do something with this, and this happens when the device block > size is 4k, and volblocksize is default, thus 8k. Mine disks sector size is > 512 native, so this is really not the case. I'm also having equal number of > disks in vdevs, and they are 5: > The AVAIL reported by zpool list doesn't account for RAIDZ overhead (or maybe it assumes optimum alignment; I can't remember). But the USED reported by "zfs list" does account for RAIDZ overhead. -alan From owner-freebsd-stable@freebsd.org Thu Apr 13 00:19:58 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 73DB3D3B6EC for ; Thu, 13 Apr 2017 00:19:58 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wm0-x236.google.com (mail-wm0-x236.google.com [IPv6:2a00:1450:400c:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0C45A7FB; Thu, 13 Apr 2017 00:19:58 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-wm0-x236.google.com with SMTP id t189so35673701wmt.1; Wed, 12 Apr 2017 17:19:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Ad64DXA4FOfQuqic3yl4a8Ua0cDI1qTZ5dkL561egos=; b=o2NNQBOjgg+n5q/FLoXCJIyCEr/jJ4g/oqz6pKyz5Bv+v/DPUgHMZnAuGTQrxushbk fq8z3SPN1CL2FBMvF5jqTtnBtXQq2DBbPR4RVoBUeRKmIHQUCzAccbSDSNYKWy/WsaGV Zc0TUtBazRsXGTSTXZ2ERgm7nAd11WIJo485ATI/vdaakJYerdrX/psGyt/8Z7sn6MH8 eyy5FpJ9+cmBUOIEkhhJh0mEYQzYmqdwD4x9o8yCn5xctNBbN63sfgbaVe5bgQVQ21O6 Rfm0BBQqKZxZ3IocadhPMHRKBuGoVtyn5dUwfJXC0bXOodonvhwdJknkBs+14X6Z+Sf1 PwzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Ad64DXA4FOfQuqic3yl4a8Ua0cDI1qTZ5dkL561egos=; b=MnYMRbGytcCj/yFX5LLIEAfTWzfQN6PugVDjfh8A21Z0XPejtkjqhpoLYchRRW2lHE 4gnzcUsds2uk4V/2J8XzGF54+REk/yT0rf+sKuh9V+OHTBn1cJfbVDEn+6gGpSWXBlD8 Xwn4s0ibUamLjrdYxasepr2/DIH7w7ymLwMMBRNCeQExX+V6siFEHI9nm1bbrRr6k9ii KZit/wIJxxT8CEOzcD/4dO/Q9SFuDnfvrhOMqQ0AYJ50GddcPJF/xAjdfMEkMUibiodH EEvGUsBaMaFzaDmfpxSHdHpKndHRXokV7Tnj+dBZeBaPglZ8rHt/1NwyD4mT82XdQbLf YFug== X-Gm-Message-State: AN3rC/4yEKpW8RzuB0wgp1FEZdHraPf2ncfdjLWVDb96mpSwp+25BT3N s5SqJA816BsUAXY7GxMbZCOWt8M5wNae X-Received: by 10.28.151.137 with SMTP id z131mr21178024wmd.138.1492042795105; Wed, 12 Apr 2017 17:19:55 -0700 (PDT) MIME-Version: 1.0 Received: by 10.28.185.66 with HTTP; Wed, 12 Apr 2017 17:19:54 -0700 (PDT) In-Reply-To: <20170406121635.99a513360faaf2f40992fb8a@freebsd.org> References: <20170406121635.99a513360faaf2f40992fb8a@freebsd.org> From: Adrian Chadd Date: Wed, 12 Apr 2017 17:19:54 -0700 Message-ID: Subject: Re: Unsupported USB BT device causes high interrupt load To: Dominic Fandrey Cc: FreeBSD Stable Mailing List Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Apr 2017 00:19:58 -0000 can you capture a usb transfer dump from usbdump ? I'd like to see if it's because the usb bluetooth code is trying to attach and failing, or whether it really is just some broken transfer / notification loop. -a On 6 April 2017 at 03:16, Dominic Fandrey wrote: > I have an Interrupt load of 2000 interrupts/s from xhci0. > > As the culprit I have identified the following device: > > ugen0.6: at usbus0, cfg=255 md=HOST spd=FULL (12Mbps) pwr=ON (100mA) > > The impression I have from googling this is that it's an unsupported > BT device sitting on board of my Intel Wireless AC 7260 card (I also > have an unsupported SD/MMC card reader sitting on the mainboard). > > usbconfig -d 0.6 power_off > > Ends the interrupt spree. > > I'm running FreeBSD 11.0-STABLE #1 r316490 amd64. > > -- > A: Because it fouls the order in which people normally read text. > Q: Why is top-posting such a bad thing? > A: Top-posting. > Q: What is the most annoying thing on usenet and in e-mail? > From owner-freebsd-stable@freebsd.org Thu Apr 13 00:21:05 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 80647D3B94F for ; Thu, 13 Apr 2017 00:21:05 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 616C2A8E for ; Thu, 13 Apr 2017 00:21:05 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 5DA2DD3B94C; Thu, 13 Apr 2017 00:21:05 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D571D3B94B for ; Thu, 13 Apr 2017 00:21:05 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wm0-x22a.google.com (mail-wm0-x22a.google.com [IPv6:2a00:1450:400c:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EA4F5A8C for ; Thu, 13 Apr 2017 00:21:04 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-wm0-x22a.google.com with SMTP id t189so35684325wmt.1 for ; Wed, 12 Apr 2017 17:21:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=K1Ia6xlIEiaVlIqSykYlLumPjSMSl/3chFR2sE2ndDo=; b=h2oxzIhgNp4m+r0dfGnKbCwDJPC/+sfcpqpfVyyr5/7+gsMiNwG+gNdtspRajOBhRT cFSvuf3b+59ECgfDh2ARM4lS26LxgIhbpMi8dftIWhFXUGWbamCXlwrEGqezzAS9pj41 zD8e9npVUG0ElUYZVqPXLOCR64K8hMRSYb1wAwT8RRiCG95ZxWJlTocKiSydVca4riBn //sT5QjzgtMUjO6Zn5bARJRiEzTekxZa3HVfsjKpmBQJPJyB0XEDTbsfrFXmNuK71NNW VDyP5x8HNtBf8ncb7hMZkg3RWgf7fIjlNjpGyzy98dt2OYrI39p1JV1OWyBJ6Vvqe/3P UJJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=K1Ia6xlIEiaVlIqSykYlLumPjSMSl/3chFR2sE2ndDo=; b=uGZjEKURKPx8K1zsLapCnwIVxVYlfozClgW2Tix5J6yAIZgLpLD4H3mNg6hL7fn452 bzBJ8v5iiIPXaIgZoEqrp3hVYsuyUc2Wv9Z5/8mCojlZ7tXr8HU6cW+tQK8FOOTTYnFQ 5fJAJrQLU0paPuDUAAvvkeybFPsgOBnRqPFm9WXSMQsDlOPsC7aIDQLeb2Zb/8ZhWyKQ gYLhy+VYtiGaPC4MTv/xDZ7fssMra0lZuUiajLIDOh76QO+575lnXOQ2GYV+Hs073rgL 59HuSiibgUUO6ezymfGIO4ZTc9V7NiT8EaAwI9IbEsp1TDPfz6RG8uji5vfF0o2oRLpn /2TQ== X-Gm-Message-State: AN3rC/4L3UodN6WmMcpAKKd7vNASDSQTvXBpK9Tfq+w7ErJatKzkOiEi 9YDAjTPFlCqUX5M9qJNMe9RQscmhhlv4 X-Received: by 10.28.0.78 with SMTP id 75mr609396wma.138.1492042863327; Wed, 12 Apr 2017 17:21:03 -0700 (PDT) MIME-Version: 1.0 Received: by 10.28.185.66 with HTTP; Wed, 12 Apr 2017 17:21:02 -0700 (PDT) In-Reply-To: <20170321164227.GE86500@zxy.spb.ru> References: <20170321164227.GE86500@zxy.spb.ru> From: Adrian Chadd Date: Wed, 12 Apr 2017 17:21:02 -0700 Message-ID: Subject: Re: Lock contention in AIO To: Slawa Olhovchenkov Cc: "stable@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Apr 2017 00:21:05 -0000 It's the same pages, right? Is it just the refcounting locking that's causing it? I think the biggest thing here is to figure out how to have pages have a lifecycle where the refcount can be inc/dec (obviously >1, ie not in a state where you can dec to 0) via atomics, without grabbing a lock. That'll make this particular use case muuuuch faster. (dfbsd does this.) -a On 21 March 2017 at 09:42, Slawa Olhovchenkov wrote: > I am see lock contetntion cuased by aio read (same file segment from > multiple process simultaneous): > > 07.74% [26756] lock_delay @ /boot/kernel/kernel > 92.21% [24671] __mtx_lock_sleep > 52.14% [12864] vm_page_enqueue > 100.0% [12864] vm_fault_hold > 87.71% [11283] vm_fault_quick_hold_pages > 100.0% [11283] vn_io_fault1 > 100.0% [11283] vn_io_fault > 99.88% [11270] aio_process_rw > 100.0% [11270] aio_daemon > 100.0% [11270] fork_exit > 00.12% [13] dofileread > 100.0% [13] kern_readv > > Is this know problem? > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Thu Apr 13 11:59:05 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1D3F4D3BE66 for ; Thu, 13 Apr 2017 11:59:05 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 0A82ECCF for ; Thu, 13 Apr 2017 11:59:05 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: by mailman.ysv.freebsd.org (Postfix) id 09D5AD3BE65; Thu, 13 Apr 2017 11:59:05 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 097AAD3BE64 for ; Thu, 13 Apr 2017 11:59:05 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C21E2CCD for ; Thu, 13 Apr 2017 11:59:04 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1cydOo-000NW0-D9; Thu, 13 Apr 2017 14:58:54 +0300 Date: Thu, 13 Apr 2017 14:58:54 +0300 From: Slawa Olhovchenkov To: Adrian Chadd Cc: "stable@freebsd.org" Subject: Re: Lock contention in AIO Message-ID: <20170413115854.GZ70430@zxy.spb.ru> References: <20170321164227.GE86500@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Apr 2017 11:59:05 -0000 On Wed, Apr 12, 2017 at 05:21:02PM -0700, Adrian Chadd wrote: > It's the same pages, right? Perhaps. > Is it just the refcounting locking that's > causing it? Don't know. > I think the biggest thing here is to figure out how to have pages have > a lifecycle where the refcount can be inc/dec (obviously >1, ie not in > a state where you can dec to 0) via atomics, without grabbing a lock. > That'll make this particular use case muuuuch faster. > > (dfbsd does this.) I can try you patch. > -a > > > On 21 March 2017 at 09:42, Slawa Olhovchenkov wrote: > > I am see lock contetntion cuased by aio read (same file segment from > > multiple process simultaneous): > > > > 07.74% [26756] lock_delay @ /boot/kernel/kernel > > 92.21% [24671] __mtx_lock_sleep > > 52.14% [12864] vm_page_enqueue > > 100.0% [12864] vm_fault_hold > > 87.71% [11283] vm_fault_quick_hold_pages > > 100.0% [11283] vn_io_fault1 > > 100.0% [11283] vn_io_fault > > 99.88% [11270] aio_process_rw > > 100.0% [11270] aio_daemon > > 100.0% [11270] fork_exit > > 00.12% [13] dofileread > > 100.0% [13] kern_readv > > > > Is this know problem? > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"