From owner-freebsd-fs@freebsd.org Sun Aug 21 00:27:54 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DEC60BC1D32 for ; Sun, 21 Aug 2016 00:27:54 +0000 (UTC) (envelope-from paul@kraus-haus.org) Received: from mail-qt0-x235.google.com (mail-qt0-x235.google.com [IPv6:2607:f8b0:400d:c0d::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A13301A37 for ; Sun, 21 Aug 2016 00:27:54 +0000 (UTC) (envelope-from paul@kraus-haus.org) Received: by mail-qt0-x235.google.com with SMTP id x25so21337541qtx.2 for ; Sat, 20 Aug 2016 17:27:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kraus-haus-org.20150623.gappssmtp.com; s=20150623; h=subject:mime-version:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=l/4c2FiAQ9OQt3nqN3I0YghSuaSP+rZ0hmVbaUJYqBI=; b=h5rQVOk8ALm4+pyANqP2qbj0oxRvFzGytXFZtwq4G+miJfd/BiXU1ymhlwFtL8CFdX 41Ah1cDQPXnG9tJNPCwT8UGz+alMsO0sigEzbvTLDSoiDLHM0a8N3aS9q8XFyzxglD86 nRtCYFyCbE8M0UU1yr8iKGxFvJ+mAKW1KkbYXFTD8qYaKhhZ79xPWNsBRgzOuGfUQIjy BZiUk1p03KwJFGJEtUN0Elte8jLWeiApSeID2cuPdDg6aHy21MLMFTPz/tuFqRdwJlKB q2Se/hTYwJn5WBW3bEuARmtMGVEqJGtYwZZsOEOG00LDWVSsIoxW1oxCtfyvXq4eXIlb +pgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:mime-version:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=l/4c2FiAQ9OQt3nqN3I0YghSuaSP+rZ0hmVbaUJYqBI=; b=O2m+PiIzJzFJ0/+AeXIJn1rEouO4e4I2UDX9Dyl9RG9JWrT14E3D5FCggF60TzZ0Ff nGf7fPsD9o+5kuj2+Bgt5KFEgWLF6mcMKws+xM4UAX2OrWp9DwM7K4qBgMLMpdRry4zt +ySRL67xb9hiORoYsRqVtkCvoEqwYAlniXIQ+GeBdAl80JFoGLrEyhTxWKBvgdn+g2nt 0V6ggpwLXXq/P64lxrB1UB2tM6papGmC8wOHxHDSNXsnmK2vIQAXjRrgnKZZKCRgqCbw YFpfXesdYXzg3cFPGR1Z4/qyKwOmyGqTCzI8nEdwjZIk+GxW2N5eOZK76FBjkAEBrM/u SGWA== X-Gm-Message-State: AEkoouupeApUxVdm+kcSaii4X32/yqjrjsqclIiVUwJks3k/qMozJqRLkZsndL+xhksVpg== X-Received: by 10.237.35.76 with SMTP id i12mr16536057qtc.41.1471739273582; Sat, 20 Aug 2016 17:27:53 -0700 (PDT) Received: from [192.168.2.133] (pool-100-4-209-221.albyny.fios.verizon.net. [100.4.209.221]) by smtp.gmail.com with ESMTPSA id l189sm7590716qke.27.2016.08.20.17.27.51 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 20 Aug 2016 17:27:51 -0700 (PDT) Subject: Re: zpool list FREE vs zfs list AVAIL Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: text/plain; charset=us-ascii From: Paul Kraus In-Reply-To: <57B59CBC.8000904@norma.perm.ru> Date: Sat, 20 Aug 2016 20:27:50 -0400 Cc: FreeBSD FS Content-Transfer-Encoding: 7bit Message-Id: <938E209E-143E-4399-809F-006E455E1C9D@kraus-haus.org> References: <57B59CBC.8000904@norma.perm.ru> To: "Eugene M. Zheganin" X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Aug 2016 00:27:55 -0000 > On Aug 18, 2016, at 7:32 AM, Eugene M. Zheganin wrote: > > What is the difference between zpool list FREE for a pool and zfs list > AVAIL ? Because they differ a lot, I'm looking at a server at the moment > where the difference is like dozens of times: zfs list reports that 97 > gigabytes is available, and the zpool list for the same pool says that > 4.18 terabytes is free. From my point of view this should be the same > number. Please provide complete zpool status, zpool list, and zfs list output. From owner-freebsd-fs@freebsd.org Sun Aug 21 06:47:19 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 30F28BC0F7C for ; Sun, 21 Aug 2016 06:47:19 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A22F717E3 for ; Sun, 21 Aug 2016 06:47:18 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [IPv6:2a02:2698:24:3eec:753f:a1e:424:e18b] (dynamic-2a02-2698-24-0-0.perm.ertelecom.ru [IPv6:2a02:2698:24:3eec:753f:a1e:424:e18b] (may be forged)) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id u7L6lDdk095587 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Sun, 21 Aug 2016 11:47:14 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1471762034; bh=YRlTr67AfL38aPIaPH3D+vNIjoscVHhtAqBa9/ekvCw=; h=Subject:To:References:From:Date:In-Reply-To; b=sc3sSP8Zn4+MW92L1YDZMulmFe/TvSZCRVfKucMsAcSi+yD7v8CjviAeLL9qFom96 hx8fT1XSt7aguzpevOFlyT5d6vMtSAeBH49v29F96fbxFZbCM0n9ZnHORX+KDDyzLH rlNepQ3whH8bGCJjMbvQ4K8vv289OxviU/4qvhKI= Subject: Re: cannot destroy '': dataset is busy vs iSCSI To: FreeBSD FS References: <57B5CD2F.2070204@norma.perm.ru> <20160819082310.GA14806@brick> From: "Eugene M. Zheganin" Message-ID: Date: Sun, 21 Aug 2016 11:47:14 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <20160819082310.GA14806@brick> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Aug 2016 06:47:19 -0000 Hi. On 19.08.2016 13:23, Edward Tomasz Napierała wrote: > Could you do "ctladm devlist -v" and see if the LUN for this file somehow > didn't get removed? > Oh, you're right - looks like ctld is holding the old clone that he used to distribute, plus the new one that is currently in its config file. What is the reason of this and how can I get rid of the old one ? Thanks. Eugene. From owner-freebsd-fs@freebsd.org Sun Aug 21 08:28:53 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 57618BC076F for ; Sun, 21 Aug 2016 08:28:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C762112E6 for ; Sun, 21 Aug 2016 08:28:52 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u7L8ShiY047556 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 21 Aug 2016 11:28:43 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u7L8ShiY047556 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u7L8Sg0f047555; Sun, 21 Aug 2016 11:28:42 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 21 Aug 2016 11:28:42 +0300 From: Konstantin Belousov To: Karl Denninger Cc: freebsd-fs@freebsd.org Subject: Re: ZFS ARC under memory pressure Message-ID: <20160821082842.GR83214@kib.kiev.ua> References: <20160816193416.GM8192@zxy.spb.ru> <8dbf2a3a-da64-f7f8-5463-bfa23462446e@FreeBSD.org> <20160818202657.GS8192@zxy.spb.ru> <20160819201840.GA12519@zxy.spb.ru> <20160820152225.GP83214@kib.kiev.ua> <97f166f0-4d47-d5a3-ecb3-d15f1ecf9c1f@denninger.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <97f166f0-4d47-d5a3-ecb3-d15f1ecf9c1f@denninger.net> User-Agent: Mutt/1.6.1 (2016-04-27) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Aug 2016 08:28:53 -0000 On Sat, Aug 20, 2016 at 11:08:44AM -0500, Karl Denninger wrote: > If you are talking about the case of an executable in which part of the > text is evicted you are correct, however, you are still choosing in that > instance to evict a page for which there will likely be a future demand > and thus require an I/O (should that executable come back up for > execution) as opposed to one for which you have no idea how likely > demand for same will be (a data page in the ARC.) No, I am not talking about only text segments. Any clean page can be reused after unmapping. > > Since the VM has no means of "coloring" the ARC (as it is opaque other > than the consumption of system memory to the VM) as to how "useful" > (e.g. how often used, etc) a particular data item in the ARC is, it has > no information available on which to decide. However, the fact that an > executing process is in some sort of waiting state still likely trumps > an ARC data page in terms of likelihood of future access. Buffer cache behaves exactly the same, since access references are not counted for the pages constituing buffers. From owner-freebsd-fs@freebsd.org Sun Aug 21 13:55:27 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF70CBC1678 for ; Sun, 21 Aug 2016 13:55:27 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7683D1CA0 for ; Sun, 21 Aug 2016 13:55:27 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id 72DAF21C8E for ; Sun, 21 Aug 2016 08:55:19 -0500 (CDT) Subject: Re: ZFS ARC under memory pressure References: <20160816193416.GM8192@zxy.spb.ru> <8dbf2a3a-da64-f7f8-5463-bfa23462446e@FreeBSD.org> <20160818202657.GS8192@zxy.spb.ru> <20160819201840.GA12519@zxy.spb.ru> <20160820152225.GP83214@kib.kiev.ua> <97f166f0-4d47-d5a3-ecb3-d15f1ecf9c1f@denninger.net> <20160821082842.GR83214@kib.kiev.ua> To: freebsd-fs@freebsd.org From: Karl Denninger Message-ID: <73545a4b-fafc-f623-a8f5-cfcd3d8a2bd7@denninger.net> Date: Sun, 21 Aug 2016 08:55:11 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <20160821082842.GR83214@kib.kiev.ua> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms040408040108020304040707" X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Aug 2016 13:55:27 -0000 This is a cryptographically signed message in MIME format. --------------ms040408040108020304040707 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 8/21/2016 03:28, Konstantin Belousov wrote: > On Sat, Aug 20, 2016 at 11:08:44AM -0500, Karl Denninger wrote: >> If you are talking about the case of an executable in which part of th= e >> text is evicted you are correct, however, you are still choosing in th= at >> instance to evict a page for which there will likely be a future deman= d >> and thus require an I/O (should that executable come back up for >> execution) as opposed to one for which you have no idea how likely >> demand for same will be (a data page in the ARC.) > No, I am not talking about only text segments. > Any clean page can be reused after unmapping. And this accounts for pages being evicted to the swap..... how?=20 (Obviously they're not simply unmapped, and thus were not "clean" when targeted by the VM) >> Since the VM has no means of "coloring" the ARC (as it is opaque other= >> than the consumption of system memory to the VM) as to how "useful" >> (e.g. how often used, etc) a particular data item in the ARC is, it ha= s >> no information available on which to decide. However, the fact that a= n >> executing process is in some sort of waiting state still likely trumps= >> an ARC data page in terms of likelihood of future access. > Buffer cache behaves exactly the same, since access references are not > counted for the pages constituing buffers. It is always amusing when I am told how things are supposed to occur (and thus assumptions that were made when writing code) when I have spent countless hours observing how they actually do behave and writing code to mitigate the very behavior that is claimed to not happen. I should have something for 11-MostlyStable on the original PR's thread in a couple of weeks, assuming I manage to complete what I'm working on right now on-schedule. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms040408040108020304040707 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjA4MjExMzU1MTFaME8GCSqGSIb3DQEJBDFCBEC9 pc0+jrZ0NX1tcdjEiw11v/KjDQbuGvjFC7TC4kl/8/Y3k5ByJDgIttFRurk1aLBW6bFxDm6l g9ZN44SP8gRCMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIADmOtAIUa 0iN5NYqGUe8eRO77avZpPHDaq7yLkFbhefpDQzRyjlqd1S8wuQikiNkXdxa8qi6ZpVBXZVvM 1hs/CtmArnxF/De3uCf2NFCSrqPAUtSfXeM7TzVLs8rtoJHnh27H706RB9BkayJXPH4GSlg1 DcSmHiGg0wPbHVN5xTirvIbUIVi09aWNnZL31wJ+rRP+zLHEVifvLb9Oxo1iODRTErO0Pi6K Odr20I2lZF/ULgGa6KuA9PG8spadfZmuDilg8WCXYdFicRPky1ldpIfVVwSEQwSUiMyzu7aF kbpYbplBvMnVu704K+mvrUJYVhHyG9TbXIj/34ICItPQFG278FuOiVVHEA1+dS7KQyOWribi 5XYg6i7EwcPw3rfZrblsOW5TDZfX+ktv57HUCBvwuKGjRHbm/MNEfRQc4npSku2aO2OmoI+K 4+zEcf6SEaJVwuEvJl2e9yb/yrDgf0zH7ESIubekgVeyzeheJ7kkFcevtJc4vsnIwXgSqwWU gsCWmnSTDdEibLq6nOAt4HrmyMd8LJ0sv1SrepoK0hoviup+NiYlL5Ytwt+L7wPW+TysIZ6t Yqu2LG7+hH6R0SLvcBRU3mMhtLvXybz39qYVFNZ+jMQOLCxkbqVi3635UchGWnA0VA7rzaRL xX0zoChmSsJDUPQED89Z6phD600AAAAAAAA= --------------ms040408040108020304040707-- From owner-freebsd-fs@freebsd.org Mon Aug 22 01:00:51 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7A62EBC1319 for ; Mon, 22 Aug 2016 01:00:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6A0851EE1 for ; Mon, 22 Aug 2016 01:00:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u7M10nHb093769 for ; Mon, 22 Aug 2016 01:00:51 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 209158] node / npm triggering zfs rename deadlock Date: Mon, 22 Aug 2016 01:00:49 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: needs-qa, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: arkadiusz@nowara.pl X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: avg@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 01:00:51 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D209158 Arkadiusz Nowara changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |arkadiusz@nowara.pl --- Comment #41 from Arkadiusz Nowara --- Hi, this bug will be fixed for 10.3 or 10-STABLE? --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Mon Aug 22 16:09:03 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 84F98BC235E for ; Mon, 22 Aug 2016 16:09:03 +0000 (UTC) (envelope-from marck@rinet.ru) Received: from woozle.rinet.ru (woozle.rinet.ru [195.54.192.68]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 197AB1BFE for ; Mon, 22 Aug 2016 16:09:02 +0000 (UTC) (envelope-from marck@rinet.ru) Received: from localhost (localhost [127.0.0.1]) by woozle.rinet.ru (8.14.5/8.14.5) with ESMTP id u7MG8jl0082678 for ; Mon, 22 Aug 2016 19:08:45 +0300 (MSK) (envelope-from marck@rinet.ru) Date: Mon, 22 Aug 2016 19:08:45 +0300 (MSK) From: Dmitry Morozovsky To: freebsd-fs@FreeBSD.org Subject: Recommended HBA for ZFS, contemporary Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-NCC-RegID: ru.rinet X-OpenPGP-Key-ID: 6B691B03 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (woozle.rinet.ru [0.0.0.0]); Mon, 22 Aug 2016 19:08:45 +0300 (MSK) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 16:09:03 -0000 Dear colleagues, what is the list of preferred disk controllers to build storage server, say 12-24 amd more 3.5" disks? Not too much IOPS are expected, and some could be mitigated with L2ARC with only metadata cached, I suppose. Two usage schemes are file server for big files (mostly cold storage) and backup/surveillance server (constant stream writes, occasional random reads) Any hints? Thanks! -- Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] [ FreeBSD committer: marck@FreeBSD.org ] ------------------------------------------------------------------------ *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru *** ------------------------------------------------------------------------ From owner-freebsd-fs@freebsd.org Mon Aug 22 16:14:54 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2F85BC252A for ; Mon, 22 Aug 2016 16:14:54 +0000 (UTC) (envelope-from juergen.gotteswinter@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B38801185 for ; Mon, 22 Aug 2016 16:14:54 +0000 (UTC) (envelope-from juergen.gotteswinter@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id 89D234C4C7B4; Mon, 22 Aug 2016 18:14:46 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fMhSK0Ap+rGd; Mon, 22 Aug 2016 18:14:41 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id 9546C4C4C135; Mon, 22 Aug 2016 18:14:41 +0200 (CEST) Reply-To: juergen.gotteswinter@internetx.com Subject: Re: Recommended HBA for ZFS, contemporary References: To: Dmitry Morozovsky , freebsd-fs@FreeBSD.org From: InterNetX - Juergen Gotteswinter Organization: InterNetX GmbH Message-ID: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> Date: Mon, 22 Aug 2016 18:14:40 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 16:14:55 -0000 LSI SAS 2008 Based HBA, there are dozens of OEM available for Budget Prices Am 22.08.2016 um 18:08 schrieb Dmitry Morozovsky: > Dear colleagues, > > what is the list of preferred disk controllers to build storage server, say > 12-24 amd more 3.5" disks? Not too much IOPS are expected, and some could be > mitigated with L2ARC with only metadata cached, I suppose. > > Two usage schemes are file server for big files (mostly cold storage) and > backup/surveillance server (constant stream writes, occasional random reads) > > Any hints? Thanks! > From owner-freebsd-fs@freebsd.org Mon Aug 22 16:16:18 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65885BC262D for ; Mon, 22 Aug 2016 16:16:18 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-oi0-x229.google.com (mail-oi0-x229.google.com [IPv6:2607:f8b0:4003:c06::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2AFDB13CB for ; Mon, 22 Aug 2016 16:16:18 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by mail-oi0-x229.google.com with SMTP id 4so158504482oih.2 for ; Mon, 22 Aug 2016 09:16:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=lK2a91ipyC3ZRcgd/+CqSRLFp8ts4E9TkPb9rV/i4G8=; b=RbsGAz3/CeAfJqEPi8RyqX+2nJp8QCVth/fCZ2xuNdBbtSmLqG5z+3oWdt5cbEsTx8 /mwEonQ6aHKzpKxZGbVV17MtEaOA8ehx3LLqdIYklfC/nR5djdk1CZbmtcuEvYnOs7sT 78s3fs0S5HIurnmkGXxpbC5BiKvyprhME1wb5nXdKz0yXJ6faHHPRUfHp64y/QkhN6rd +NMJ7NBQuOHqlTtxLzn07JztSqlH+IyTS8fQfRVQaFAucg18gxU3EEjR6VFZh18BjAak W4xrkBZksbh18dGSgUIie12cPY3NWqa6C7ITv1YFv5R7VSBAXjG7flxUhijl6PL89EfQ PfJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=lK2a91ipyC3ZRcgd/+CqSRLFp8ts4E9TkPb9rV/i4G8=; b=AhpGq69tlr+XsrdZdBPrQWxEV19KjS4BWE+A4TKtP9FfnNeq07nmkJkhIZp8Q7Pr9E f08mXP7R++jMkTuZIOBcc8T1+0Yt7HTE3meAUEXS+rDBdgUTEV7S72TavLIcZp5SnIee rdAKa4ecDhdSkRnQziZLRpj35LWv5jECebzREEol1IoPKBXmslYFFhukih5+Yv+SmMJW QD5HFD6jNUm5xoNknl2k2t4UyKWJ/H1SzgpqdyzGvkRgizqTbQDVh02tF8j+D6oqJXJu PBG5Jk3W75zu76QVSICP9nUBef+1MLBOEekXUCNNGniiwor1kJZBw25JA/AqoeFJzckI ECyg== X-Gm-Message-State: AEkoouuWxW5YBoPfu95gRuO6hrZ7yw4G5o0zR1GWeoxeKfRLNooh+HZrKvdyhPsbqME0MCPa9YvqWtQULs6GAg== X-Received: by 10.202.252.135 with SMTP id a129mr13227183oii.4.1471882577432; Mon, 22 Aug 2016 09:16:17 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.237.68 with HTTP; Mon, 22 Aug 2016 09:16:16 -0700 (PDT) In-Reply-To: References: From: Freddie Cash Date: Mon, 22 Aug 2016 09:16:16 -0700 Message-ID: Subject: Re: Recommended HBA for ZFS, contemporary To: Dmitry Morozovsky Cc: FreeBSD Filesystems Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 16:16:18 -0000 On Mon, Aug 22, 2016 at 9:08 AM, Dmitry Morozovsky wrote: > Dear colleagues, > > what is the list of preferred disk controllers to build storage server, s= ay > 12-24 amd more 3.5" disks? Not too much IOPS are expected, and some coul= d > be > mitigated with L2ARC with only metadata cached, I suppose. > > Two usage schemes are file server for big files (mostly cold storage) and > backup/surveillance server (constant stream writes, occasional random > reads) > > Any hints? Thanks! > =E2=80=8BWe've had great experience with LSI (now Avago) 9211-8i and 9211-8= e controllers. These use the older LSI2008 chipset, fully supported by the mps(4) driver under FreeBSD 9+. We use these in 24-bay, 4U SuperMicro chassis with a local motherboard (all-in-one server). And in multi-chassis setups with 90+ hard drives (2U SuperMicro chassis with the motherboard and HBAs connected to 4U SuperMicro JBOD chassis). --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@freebsd.org Mon Aug 22 16:18:48 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BC039BC2746 for ; Mon, 22 Aug 2016 16:18:48 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x22a.google.com (mail-wm0-x22a.google.com [IPv6:2a00:1450:400c:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4AAB31511 for ; Mon, 22 Aug 2016 16:18:48 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x22a.google.com with SMTP id q128so128185776wma.1 for ; Mon, 22 Aug 2016 09:18:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=rktJlOJ+wNc7wrKeT92Y7AoK2HmOPIIQHwPNcoSvDUw=; b=GPh4g7PgKBq5HTXa0dlwXlh9u5tqZsxWVS1SvZ4x2T//t7ro/Io+QMFhTqnVUCSgVw vW/tf5b6gQaQ8GipDrf2ZeBN7eSy4gDtiulNUzdzuYh5SJa29KsBCssBgwhLrStP7csO 8XGGCo1XxqcOKamUGMmajz5JguPjzcFRXgAKEFMietRvlqpnvGpqvvK2zxiUCRXf5yXi OmnYggpAqAyG5pYMZ8vktJUM8Yep3iuv19g6tGaM4Cf8vtsEryap9qfLdcdJG3n+xpK0 GaVS2KuvWDkjYkb5wuDBsWB3Q3vzqFIOorp+ktNEEx5l4WabWJ2j8L25rc4J5Z8kw0ly +DGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=rktJlOJ+wNc7wrKeT92Y7AoK2HmOPIIQHwPNcoSvDUw=; b=eueiX9lF0zhQgMIErnp+ce5vBPbVu741U4z/iafO+1dcssM8ZbvPEnqEcX2C3MxvhF O7aaoM6U+X98IUad9k1DZVyee0dbBo+8+pt9OhYQCzNp7xaDHncWFSSEgMRCMsXh1c4v t4cJ10+Cih+H74uAZpN52zh+9FDde7HV6a0cyjiUqOZi8Wt/WNLG492w55LGZA8kM61x 6XvYhoPv3DqnfoLfPiLbDDJFUtcGs9NenHxvFsrTqDXqWUa7rQIOr8xd5SEQdgps733F 2sxKo5pQnO6eDVW435i1KsE3Pq4IcQckIWtR6OEbCw4sd5FjtK6lZzIvg/NZfrVbJO7u 4XfQ== X-Gm-Message-State: AEkoouuMtB9Hdrhrn8ZBQkHlnthyeNMJHtClTpSvAElEBgBfAwGDEpyxOMEQ9U7u4rQ3+A== X-Received: by 10.28.158.142 with SMTP id h136mr15528576wme.10.1471882725730; Mon, 22 Aug 2016 09:18:45 -0700 (PDT) Received: from macbook-air-de-benjamin-2.home (ANice-651-1-185-161.w83-197.abo.wanadoo.fr. [83.197.121.161]) by smtp.gmail.com with ESMTPSA id f187sm22348584wmf.15.2016.08.22.09.18.44 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 22 Aug 2016 09:18:45 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: Recommended HBA for ZFS, contemporary From: Ben RUBSON In-Reply-To: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> Date: Mon, 22 Aug 2016 18:18:44 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> References: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> To: freebsd-fs@FreeBSD.org X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 16:18:48 -0000 > On 22 Aug 2016, at 18:14, InterNetX - Juergen Gotteswinter = wrote: >=20 > LSI SAS 2008 Based HBA, there are dozens of OEM available for Budget = Prices With the IT firmware. > Am 22.08.2016 um 18:08 schrieb Dmitry Morozovsky: >> Dear colleagues, >>=20 >> what is the list of preferred disk controllers to build storage = server, say=20 >> 12-24 amd more 3.5" disks? Not too much IOPS are expected, and some = could be=20 >> mitigated with L2ARC with only metadata cached, I suppose. >>=20 >> Two usage schemes are file server for big files (mostly cold storage) = and=20 >> backup/surveillance server (constant stream writes, occasional random = reads) >>=20 >> Any hints? Thanks! >>=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Mon Aug 22 16:22:53 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D8AA4BC29C7 for ; Mon, 22 Aug 2016 16:22:53 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 91E9B19BB for ; Mon, 22 Aug 2016 16:22:53 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id 86C6721CFD for ; Mon, 22 Aug 2016 11:22:51 -0500 (CDT) Subject: Re: Recommended HBA for ZFS, contemporary To: freebsd-fs@freebsd.org References: From: Karl Denninger Message-ID: Date: Mon, 22 Aug 2016 11:22:42 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms000000040104070106000601" X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 16:22:53 -0000 This is a cryptographically signed message in MIME format. --------------ms000000040104070106000601 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable I'm still a fan of the LSI SAS-9211 cards along with (for higher density) a SAS port expander. I've been extremely happy with this combination with one caveat -- most systems must boot from the actual card and not a port on the expander(s) you connect to it due to BIOS constraints. They're also crazily cost-effective on-balance. In practice this means you have 4 ports on the base card available (2 used for mirrored boot drives, 2 for other things) and the rest of the disks go on the expander, or you run more than one card. Whether you can actually saturate this combination depends on what you attach. In large configurations, especially those stuffed with SSDs, you can -- at which point something faster such as the 93xx series which are materially faster (but still in "host" mode as opposed to any sort of "smart", "raid" or "buffered" mode) should be considered. The "smarter" the card the dumber the results tend to be when ZFS is in use, in my experience :) Pay attention as well to the slots on your server board and the number of lanes they can support. It makes no sense to buy a fast board that your motherboard cripples because it cannot run it in the slot(s) you have available at full speed. On 8/22/2016 11:08, Dmitry Morozovsky wrote: > Dear colleagues, > > what is the list of preferred disk controllers to build storage server,= say=20 > 12-24 amd more 3.5" disks? Not too much IOPS are expected, and some co= uld be=20 > mitigated with L2ARC with only metadata cached, I suppose. > > Two usage schemes are file server for big files (mostly cold storage) a= nd=20 > backup/surveillance server (constant stream writes, occasional random r= eads) > > Any hints? Thanks! > --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms000000040104070106000601 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjA4MjIxNjIyNDJaME8GCSqGSIb3DQEJBDFCBEAy v3fhZ6StHQV6OPN/zRimbislMyeA4CYkRRPq1lp5rZTeeR/g7Cu7xjMHWkJxCiSrFXYPmEty 5xHq501xI2uwMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIAK/4JUMWH tJZ8rZF4vpZwLkcbZfD9i2UPeOqBU53MYXuwLMJIlwaDOcgUYyJ8Kgth5JbA55pBkz3BINZb JhMt4c7JNycisV+FcYiBv8AnJC5Wm6FTBFLrOnSBkGowZg777LXD9AhYH7f9KyYkTqtWPVHY 7ed7iEb2jWu8utjVV4iOjnWqmCCGKVLbrV/4uwlS0vinf2oXf/2/rqygCspa7sz0o0vb9obV u3V0Oh4vVrrq6SVJiOsJVllG2ZFvCDhhvhYi8wmRHmiTT02r+xrMXVCfBNME4Z1Zv+AKwEC0 BKLjk5/XHWhnahO3l3SpFrG38Fyi7Db8rFhYR4RLv7EcOCpHYJ2nq0wZJ4uhNlio9I8uM5Rx PQ5s9lNrIaq4TnXvd57msoTXCCjWY7vvinQPJkG4gsHBm3cV8jruR0nb7ZQdmHekJ5zfFuDF wzlLS4I77/wy43ez4zPX6xyr8vXZJgXy1KFNMybXLN4CFnR4IN2Zo2/siokPskzDclLMMCa2 2VIC35Mmm9pJ7itdN6kKYkl3BkSI4fxouu0xxjoVWLGpiL9kXu5ioFJTjfnmkvMzBwxD7bvr GhhSyefm9/Fo4vYZxXQGsq0Q4RopdUHYofNLKlELagGP/MccMSr1AGufrXdtJICvwoV7JR6C qr8NqyOeLtZ4bcvQNoRr4R9Zd38AAAAAAAA= --------------ms000000040104070106000601-- From owner-freebsd-fs@freebsd.org Mon Aug 22 16:29:03 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2FA1ABC2C06 for ; Mon, 22 Aug 2016 16:29:03 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-oi0-x22a.google.com (mail-oi0-x22a.google.com [IPv6:2607:f8b0:4003:c06::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E85351DF8 for ; Mon, 22 Aug 2016 16:29:02 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by mail-oi0-x22a.google.com with SMTP id l203so158853534oib.1 for ; Mon, 22 Aug 2016 09:29:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=lyu/f3PX6sLXwrSTk8kF8u7qiHJhOr8VzqU12NvPg70=; b=STfUrhPAwBG69kgqqsKMasE2k50qLNAZYAQpqfm149GiXUIfWZVxSJehI/1NtJh2uY jro1DM8oQJg2vSMkd61uaZXL8LuesiU0jYWTLn9kmexsWbucW8sbJ9C93WyTGkhEbTRJ QFEf1TskrNiRwAWnUgbHPtgwPLULOxBLPV0vAoKeELZZJgqYNZnXwZeLEhugG6TcmNYh 7cMD45lgNnNJ8uzcVevIJO2fLdtaLoiVPwMyJnmFxrYk+pPK+d6+XLYF16mB02DFbIEL xDtLKy/qga7A0S08QDxLpGDh3MjcnWu1QhXCWN1XSsAYDvz8fCS7h1/giz3TsBtiSmlc Y1JQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=lyu/f3PX6sLXwrSTk8kF8u7qiHJhOr8VzqU12NvPg70=; b=jGd5w7gGo0P9j3DVbQ4amgLY04OglcnNdYXBC4ZmrfQUJbJ164EW+XhlP4bM0f40Wq GhuuWR4nkkQl4AQUus5y8WgBWz60hwwNuH1O6xYF0nryW934jhfKyE77b+Sy6rhyjyop omQnEZgw98oxHhTh0WlSCv+/HYodDXyiO60vkiXbhZ++QPRPGuB0ygUFYhxBSzM2gPwy 0jbcvmoAveH95/+Z1YAoatxjJuEmEER5K7VGX1iHRjkAWM6rMI+5phHbESH1D3mW2rjJ XkmidwVyuPDdTYXh0dyNpgZa1khfHTMJB2KRPx0sz1pef8V1oJ/OStQRZJ+eYLpNS4wn xElQ== X-Gm-Message-State: AEkoout7gNzIbZlBZRBfMij06K5/2C1A2OApidaSWDUgNqVnqO2MnfMaqdVQb8wRC91+KyvB3ZdFGPGXP+Tcfw== X-Received: by 10.157.35.49 with SMTP id j46mr14941205otb.38.1471883342295; Mon, 22 Aug 2016 09:29:02 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.237.68 with HTTP; Mon, 22 Aug 2016 09:29:01 -0700 (PDT) In-Reply-To: References: From: Freddie Cash Date: Mon, 22 Aug 2016 09:29:01 -0700 Message-ID: Subject: Re: Recommended HBA for ZFS, contemporary To: Karl Denninger Cc: FreeBSD Filesystems Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 16:29:03 -0000 On Mon, Aug 22, 2016 at 9:22 AM, Karl Denninger wrote: > I'm still a fan of the LSI SAS-9211 cards along with (for higher > density) a SAS port expander. I've been extremely happy with this > combination with one caveat -- most systems must boot from the actual > card and not a port on the expander(s) you connect to it due to BIOS > constraints. They're also crazily cost-effective on-balance. > > In practice this means you have 4 ports on the base card available (2 > used for mirrored boot drives, 2 for other things) and the rest of the > disks go on the expander, or you run more than one card. > =E2=80=8BAlternatively, you can use the SATA/SAS ports on the motherboard f= or the OS drives / to boot from, leaving all the ports on the HBAs for the data disks. :) We also use the motherboard ports for the ZFS Log and Cache devices, to separate the I/O channels. Granted, this depends on the motherboard having good quality SATA controllers, but most are good enough for booting/loading programs from. > The "smarter" the card the dumber the results tend to be when ZFS is in > use, in my experience :) > =E2=80=8BThat's a great way to explain things. Will have to remember that = one!=E2=80=8B --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@freebsd.org Mon Aug 22 16:30:26 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 771C7BC2C7A for ; Mon, 22 Aug 2016 16:30:26 +0000 (UTC) (envelope-from marck@rinet.ru) Received: from woozle.rinet.ru (woozle.rinet.ru [195.54.192.68]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0DF031EA5 for ; Mon, 22 Aug 2016 16:30:25 +0000 (UTC) (envelope-from marck@rinet.ru) Received: from localhost (localhost [127.0.0.1]) by woozle.rinet.ru (8.14.5/8.14.5) with ESMTP id u7MGUJ0Z082983; Mon, 22 Aug 2016 19:30:19 +0300 (MSK) (envelope-from marck@rinet.ru) Date: Mon, 22 Aug 2016 19:30:19 +0300 (MSK) From: Dmitry Morozovsky To: Karl Denninger cc: freebsd-fs@freebsd.org Subject: Re: Recommended HBA for ZFS, contemporary In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-NCC-RegID: ru.rinet X-OpenPGP-Key-ID: 6B691B03 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (woozle.rinet.ru [0.0.0.0]); Mon, 22 Aug 2016 19:30:19 +0300 (MSK) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 16:30:26 -0000 Karl, On Mon, 22 Aug 2016, Karl Denninger wrote: > I'm still a fan of the LSI SAS-9211 cards along with (for higher > density) a SAS port expander. I've been extremely happy with this > combination with one caveat -- most systems must boot from the actual > card and not a port on the expander(s) you connect to it due to BIOS > constraints. They're also crazily cost-effective on-balance. > > In practice this means you have 4 ports on the base card available (2 > used for mirrored boot drives, 2 for other things) and the rest of the > disks go on the expander, or you run more than one card. > > Whether you can actually saturate this combination depends on what you > attach. In large configurations, especially those stuffed with SSDs, > you can -- at which point something faster such as the 93xx series which which immediately leads me to the question: what is the driver supporting 9300 in HBA mode? Quick googling does not reveal interoperability, and 9300 is not in hardware list for FreeBSD yet. > are materially faster (but still in "host" mode as opposed to any sort > of "smart", "raid" or "buffered" mode) should be considered. > > The "smarter" the card the dumber the results tend to be when ZFS is in > use, in my experience :) Yes, I'm well aware of it ;) Thanks! -- Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] [ FreeBSD committer: marck@FreeBSD.org ] ------------------------------------------------------------------------ *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru *** ------------------------------------------------------------------------ From owner-freebsd-fs@freebsd.org Mon Aug 22 16:31:33 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 527F0BC2D8F for ; Mon, 22 Aug 2016 16:31:33 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x244.google.com (mail-wm0-x244.google.com [IPv6:2a00:1450:400c:c09::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EF717116E for ; Mon, 22 Aug 2016 16:31:32 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x244.google.com with SMTP id q128so14254875wma.1 for ; Mon, 22 Aug 2016 09:31:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-transfer-encoding:subject:message-id:date:to :mime-version; bh=95xYU+/KKGl1zoTz6Vv2lAw4iCm3M1i5dL3BYzd8OLA=; b=Va5UHdlp0b4fuDYo6Qwqpfeb3VwqXcYos26Z1khBbGrbZgffZaD6zDG8+W+DjCTKYi dQs8sUvnqW1V6ouLRqIDmGpgjjPvE8ibKOJWlmnyH8fZgoPj2k3R4e+ceD8+3SzJ1LTZ GKBVuywqsDTDLDF5WJhIgyoto867OEFxAHODZwt9IBc3OgkcUTxNPRNgG31vxPwfRODf gTgrPwYs1VJ8wlPv60Vy3KUho6XRuZyzUVQnIqrO5XMH8Xn7sHD3Hoy1s2Z9T7Xpq+lY oiGhPozgE2H9QbPuMEbC9DjPF8I1l/o41ASZyx9Xpxofxh8OIddeVMnUgqkJn+1mfItf IRIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:content-transfer-encoding:subject :message-id:date:to:mime-version; bh=95xYU+/KKGl1zoTz6Vv2lAw4iCm3M1i5dL3BYzd8OLA=; b=Niqlr/lDM2dpDj0yNbNKSIdVgaNBAG6AH8xG8q5Pf4Jxa++Z1udJSW3ZtZLbJxHqNy Wb+F8kuhovKpV49BRaIVRM5/SPrFXYv8Bnzh9QVTgU+bY93oMSu+pUELJ3BU90qJfH8x pXowTCozZ56GBD3V+rhZyZrfwXoZydPzEl7dS9LtOKyv2hbr5tjB12fWrrBLAfei0r0k g1dCae5XrEZRbitCljQNgCUOiYHJ6a3J+7MfbJlhLsClqQXDYli11Ug8emNHQSLGQ5xM cUbPDVUnh+csy9117UaCBwbHRaFwh8bRtKIDxNGy3NtuIkVzrs3ufZ9iXBdVEReE4j2u hCww== X-Gm-Message-State: AEkooutx0ZQRvRG8heF73A8eSwLgaa0IJ2ywedgXBLx9HrosMaEDwJWj5DAmK5TPfCy4dg== X-Received: by 10.194.175.106 with SMTP id bz10mr21047044wjc.42.1471883491293; Mon, 22 Aug 2016 09:31:31 -0700 (PDT) Received: from macbook-air-de-benjamin-2.home (ANice-651-1-185-161.w83-197.abo.wanadoo.fr. [83.197.121.161]) by smtp.gmail.com with ESMTPSA id hy3sm24987045wjb.8.2016.08.22.09.31.30 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 22 Aug 2016 09:31:30 -0700 (PDT) From: Ben RUBSON Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: [ZFS] "online" spares ? Message-Id: Date: Mon, 22 Aug 2016 18:31:29 +0200 To: FreeBSD FS Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 16:31:33 -0000 Hello, Let's assume the following status : pool: zroot state: ONLINE config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 (...) spares 15751675431801904320 REMOVED was = /dev/label/X92020TYEEGHT For some reason one of the spares is in removed state but is now = available again to the system. "online" command does not work for spares. Is there any way to bring spares back to available state without having = to "remove" and "add" them again ? Many thanks ! Best regards, Ben From owner-freebsd-fs@freebsd.org Mon Aug 22 16:46:46 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A417DBC21DB for ; Mon, 22 Aug 2016 16:46:46 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5F2741EAF for ; Mon, 22 Aug 2016 16:46:46 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id 0731521E30 for ; Mon, 22 Aug 2016 11:46:44 -0500 (CDT) Subject: Re: Recommended HBA for ZFS, contemporary To: freebsd-fs@freebsd.org References: From: Karl Denninger Message-ID: <580a6ea8-9c84-3e99-5a7d-7b2434c84e67@denninger.net> Date: Mon, 22 Aug 2016 11:46:35 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms020807000804010108050705" X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 16:46:46 -0000 This is a cryptographically signed message in MIME format. --------------ms020807000804010108050705 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 8/22/2016 11:30, Dmitry Morozovsky wrote: > Karl, > > On Mon, 22 Aug 2016, Karl Denninger wrote: > >> I'm still a fan of the LSI SAS-9211 cards along with (for higher >> density) a SAS port expander. I've been extremely happy with this >> combination with one caveat -- most systems must boot from the actual >> card and not a port on the expander(s) you connect to it due to BIOS >> constraints. They're also crazily cost-effective on-balance. >> >> In practice this means you have 4 ports on the base card available (2 >> used for mirrored boot drives, 2 for other things) and the rest of the= >> disks go on the expander, or you run more than one card. >> >> Whether you can actually saturate this combination depends on what you= >> attach. In large configurations, especially those stuffed with SSDs, >> you can -- at which point something faster such as the 93xx series whi= ch > which immediately leads me to the question: what is the driver supporti= ng 9300=20 > in HBA mode? Quick googling does not reveal interoperability, and 9300= is not=20 > in hardware list for FreeBSD yet. So much for LSI's claims eh? :-) >> are materially faster (but still in "host" mode as opposed to any sort= >> of "smart", "raid" or "buffered" mode) should be considered. >> >> The "smarter" the card the dumber the results tend to be when ZFS is i= n >> use, in my experience :) > Yes, I'm well aware of it ;) > > Thanks! Incidentally actually managing to saturate a 9211 (which comes up as a "SAS-2008") on the mps driver is non-trivial. If you THINK you are, make sure you really are. If you really are and have a second PCIe slot with the requisite lanes available that's probably the next thing to do. Yes, it's possible (especially with SSDs) to saturate these but not easy. You can get the 2xxx series cards in 16-port versions (4 x 4) which leaves you three ports that can each be attached to an expander.=20 The limiting factor usually winds up being the configuration of said plugs and the shelves for the drives themselves (2i/2e configurations work great if you're using expansion chassis with an expander in them, by the way.) I have a number of the Intel 6-port expanders in production servers and have never have had any trouble with them; they just plain work. The only note I'll make on the expanders (and these host cards) is to make sure you have sufficient airflow in the chassis; they're passive heat-sink cooled so airflow matters. In any sort of proper server chassis this is probably not going to be an issue but if you're going for something that doesn't sound like an old 707 jet on take-off you need to pay attention to make sure there's sufficient cooling airflow. I tend to avoid using the motherboard SATA ports; I've had my share of "fun" with them over time, especially if I need to hot-swap a drive (e.g. a failed member of the root mirror.) You only need to have a machine crash on you unnecessarily in this fashion once to be disabused of that sort of insanity in the future. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms020807000804010108050705 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjA4MjIxNjQ2MzVaME8GCSqGSIb3DQEJBDFCBEAs 8TNvA+Elyw/z6MtJuqDGeJs537cc2nIunj5GvivQCw1cnmCiBM+rsceoBGC/V8pvISDoJ3Ei HENN29GvVgvHMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIApwc440FT OxifqYGYVuCr4kmfZVGNl0oQ/oXjt8nEJOeUWq4CRe+MwJELzY+3/wHZjTuWVML3BIqeHpGO 4wrxm6fECCoLNeiAvW960wCuUOm5wlV+DDQ2q9TAgfOchM//syRjyW1xigw4+Mus2aUmYRtJ +1hB9IeMaYg5PgH16lfCz4hIV/0NhOlbPy9+qk8QPqasuZaJ1Nq+sEiJ8znZ+Z3SK2NwUhmN +QJW0x7byBsud1bm0Ay3OJAD5MUxkq1fwqRan/8bipAiAnNVN5PIWpBpVLD1AhckqxoLYecb WPbzJsGKfFqaiLzGKGOzqoHz8SDX7hP21ePoaRkQfjh/jp3wvBU6rMcJpRhzyWRhwgX+kcJm 2/UqbUxyqcBE5SC9gJyz+YgYDThlEuZ3QY1vxlpPtA5IMztaQo3g09RLCdF1NvCSbUWeM/u5 Fh9FEsXsYowQ3P65laA381RBzEFFXeSTKnbXDeoGUURcCe3/xHL2zPYthq7CLC/kCE7QhrSa QCt3SHuZ2wvh4g1UNrGCH5myx+YxkAecxRRemf77CAE2I+EeWVRD1J9S3Pv3UsYLYik2MPox TiL3Vt7/QrU5zOQi0Whi2HGKeg8JH9ozZpyciN+l1obukTxuxypHsRel1PmWI/+CNi8A0b8N V2KCEoqz3xbxE5mJEbvQiUrVJ3EAAAAAAAA= --------------ms020807000804010108050705-- From owner-freebsd-fs@freebsd.org Mon Aug 22 17:13:45 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0106BC2BC0 for ; Mon, 22 Aug 2016 17:13:45 +0000 (UTC) (envelope-from lkateley@kateley.com) Received: from mail-it0-x22b.google.com (mail-it0-x22b.google.com [IPv6:2607:f8b0:4001:c0b::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6E0AD1F95 for ; Mon, 22 Aug 2016 17:13:45 +0000 (UTC) (envelope-from lkateley@kateley.com) Received: by mail-it0-x22b.google.com with SMTP id f6so90241012ith.0 for ; Mon, 22 Aug 2016 10:13:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kateley-com.20150623.gappssmtp.com; s=20150623; h=reply-to:subject:references:to:from:organization:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=J0Pkp/cO33U4+OXWddjFwZqWgJ/CD2CLNYVXZCxKlxk=; b=G8L7ZhPMlHMUCTBIcTye/b014bdE4fYCn7CkT2bpwkyseTaSbWNpmVhQOF/MEv3TU8 Ub2DQETiV4D8EHuaPVMI1G5RXvAikCZ8C7OhLF3lvZSLizB05IWReegKw4jQP4udR85W AA3MI8uhwg/RS8TA/I2lOzW8bXb09RcnE1Zjn63YKQDnF85q8B4L3VAwZRITKCpCiypu V6NoX3vobDAj6fflxWK9zq3syezY4wZndOjVcehICNO0C6YVBI1gaVLpSZoLBb3ZOwaE wmHeAKSRPd+NMt1qxxq/CbnpGWQp1pnU/82s61+GJwwt4yOZNWDqgqqDNSSO3z+ktIwT uZHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:reply-to:subject:references:to:from:organization :message-id:date:user-agent:mime-version:in-reply-to :content-transfer-encoding; bh=J0Pkp/cO33U4+OXWddjFwZqWgJ/CD2CLNYVXZCxKlxk=; b=Ui+fA7i/YyL+RrIh5pdLHhUh5jfU0a9dwpbMASLq6MQpWvyhO4t1RHCFhuVaL/64fF TR7ey3CQQbPZKc9dBEoumwjsXGFWFbPaAKn2LnAPZ1iLPJ1uZXvkwOkoiYTYq9/3QV/X 9NeCtutO44h4YhaJ+D9k3/Dm9tK5yCAN8gwueTYzwqQvQYGsGs2xRUtq2EmJjJe/iGnv 7ELnp2X4btat619xYpZX1yO3mzMXXw/6fdeLJ4qkkg1gMxCF18FNYzi53LyAMpD4d0w8 iuZMqidm7jDE3g206Iw5jSCJb7jHh7+tkXxmLqQP7B3xLFXzQbesT89sw7KunFUC551Y CaHA== X-Gm-Message-State: AEkooutStLOwfIsMrFOiOvYeaFdFmHje5VU73EkEehUiAXFrmg+HWOOO7rjRRw1luv2VEQ== X-Received: by 10.36.57.215 with SMTP id l206mr23229227ita.5.1471886024648; Mon, 22 Aug 2016 10:13:44 -0700 (PDT) Received: from Kateleyco-iMac.local (c-50-188-36-30.hsd1.mn.comcast.net. [50.188.36.30]) by smtp.googlemail.com with ESMTPSA id i67sm6827035itf.3.2016.08.22.10.13.44 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Aug 2016 10:13:44 -0700 (PDT) Reply-To: linda@kateley.com Subject: Re: Recommended HBA for ZFS, contemporary References: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> To: freebsd-fs@freebsd.org From: Linda Kateley Organization: Kateley Company Message-ID: <059af3ed-71b3-b31d-893c-732121989cee@kateley.com> Date: Mon, 22 Aug 2016 12:13:43 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 17:13:45 -0000 Not just IT firmware, but you need to make sure the driver and firmware match. I try to use 16 if possible, 20 works but I have only found the 20.00.04.00 version of firmware to be reliable. Would love to hear if people have different experience. I am about to try the rocket raid in IT mode because I am tired of fighting with lsi firmware. Very very stable when you get the right combo, 1000's of phantom checksum errors if not. I am starting to like 12g 3008 cards better as the firmware seems to be a little more stable at version 9. Been testing 12 and so far so good, but haven't run my larger stress tests yet.. I would definitely go to the avago site and make sure there is driver and firmware for the card you are purchasing before you buy. Linda On 8/22/16 11:18 AM, Ben RUBSON wrote: >> On 22 Aug 2016, at 18:14, InterNetX - Juergen Gotteswinter wrote: >> >> LSI SAS 2008 Based HBA, there are dozens of OEM available for Budget Prices > With the IT firmware. > >> Am 22.08.2016 um 18:08 schrieb Dmitry Morozovsky: >>> Dear colleagues, >>> >>> what is the list of preferred disk controllers to build storage server, say >>> 12-24 amd more 3.5" disks? Not too much IOPS are expected, and some could be >>> mitigated with L2ARC with only metadata cached, I suppose. >>> >>> Two usage schemes are file server for big files (mostly cold storage) and >>> backup/surveillance server (constant stream writes, occasional random reads) >>> >>> Any hints? Thanks! >>> >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Mon Aug 22 17:18:13 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4A98FBC2CD5 for ; Mon, 22 Aug 2016 17:18:13 +0000 (UTC) (envelope-from lkateley@kateley.com) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0FF411337 for ; Mon, 22 Aug 2016 17:18:13 +0000 (UTC) (envelope-from lkateley@kateley.com) Received: by mail-it0-x236.google.com with SMTP id x131so98151395ite.0 for ; Mon, 22 Aug 2016 10:18:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kateley-com.20150623.gappssmtp.com; s=20150623; h=reply-to:subject:references:to:from:organization:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=Y65/l5jd/0HRhUK74lgkLxxyG2ePNxzXHj0B1I8Ktc8=; b=T7I5WDStt7mn0PszK7M8EQj/801Oxk5DHfwk7eS9AaRMDi8etWkUDbWJktQ+R0Uw6Q DGb+ADpLK2md0ZGcsK2zTTyaXi1SsLEH6yxO5tlYl64b5ukRUwva/ZVM4sm7wq7XFYPk VzW5GPZSAHiq9xPP0DMnmJdf/FLsXC7QtAsxaXahsu5Rx5ZEDPHZv9A3s6s1Tv9JbvtN OGxZaXfC+vrIxNkiJow8o8uGYjSVwDtNgbpHygX+C5F3C25XBh0DaRA0Iw6zLLJmvqM2 tf4yrcxysmkMbEZIt2apREZnw2W36isfC1HIyfgmXMCMcByp+maYdx8/9dq0HYgqK/Xn TyDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:reply-to:subject:references:to:from:organization :message-id:date:user-agent:mime-version:in-reply-to :content-transfer-encoding; bh=Y65/l5jd/0HRhUK74lgkLxxyG2ePNxzXHj0B1I8Ktc8=; b=cU6FvG0EbFd2jLuCsSGs3KvBrl51k/nYpByCYlJoe0kjD4JIirTxmHyHIyuAe8SZLQ 6WxO/594QrwO1t5vXpcuA+itNz5M7fExhpQqwCkiH67AN1UBNC1iPzrt1UUcTZd6ZvpO 7TA3YePzuZhF7vJMmPuJkb3I0q4bfPKENG9qpwShh/oa9vvYZzpH6Fe7w/kCo0I+XhaB c6ZDWAhRj7TGzBIIupbz1HYtXKwhrv/jP9Wx1wGlxzSR3Hdex5GqTNXkTVfGTsq0qQhu 7DyGYd2i6aIl+wuiOsHTlDFV5ZoVh21EYRENhazuF/1+8AqmRAeEnkdbT9eOHG4d3j+o gHgQ== X-Gm-Message-State: AEkooutbv5kpwwVVO8g6Zzt68Cm2rUOoWpnpHUzYqkcwR8duDA9D6ECCIpcLAqOEAhwimQ== X-Received: by 10.107.200.212 with SMTP id y203mr24726324iof.187.1471886292425; Mon, 22 Aug 2016 10:18:12 -0700 (PDT) Received: from Kateleyco-iMac.local (c-50-188-36-30.hsd1.mn.comcast.net. [50.188.36.30]) by smtp.googlemail.com with ESMTPSA id o15sm11292323ith.2.2016.08.22.10.18.11 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Aug 2016 10:18:11 -0700 (PDT) Reply-To: linda@kateley.com Subject: Re: Recommended HBA for ZFS, contemporary References: To: freebsd-fs@freebsd.org From: Linda Kateley Organization: Kateley Company Message-ID: Date: Mon, 22 Aug 2016 12:18:11 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 17:18:13 -0000 On 8/22/16 11:30 AM, Dmitry Morozovsky wrote: > Karl, > > On Mon, 22 Aug 2016, Karl Denninger wrote: > >> I'm still a fan of the LSI SAS-9211 cards along with (for higher >> density) a SAS port expander. I've been extremely happy with this >> combination with one caveat -- most systems must boot from the actual >> card and not a port on the expander(s) you connect to it due to BIOS >> constraints. They're also crazily cost-effective on-balance. >> >> In practice this means you have 4 ports on the base card available (2 >> used for mirrored boot drives, 2 for other things) and the rest of the >> disks go on the expander, or you run more than one card. >> >> Whether you can actually saturate this combination depends on what you >> attach. In large configurations, especially those stuffed with SSDs, >> you can -- at which point something faster such as the 93xx series which > which immediately leads me to the question: what is the driver supporting 9300 > in HBA mode? Quick googling does not reveal interoperability, and 9300 is not > in hardware list for FreeBSD yet. The avago website has driver firmware combos for all of the different cards. The only ones that are troublesome are the onboard supermicro. I was told by an SE at avago that you need to go to supermicro for the oem firmware. ftp.supermicro.com > >> are materially faster (but still in "host" mode as opposed to any sort >> of "smart", "raid" or "buffered" mode) should be considered. >> >> The "smarter" the card the dumber the results tend to be when ZFS is in >> use, in my experience :) > Yes, I'm well aware of it ;) > > Thanks! > > From owner-freebsd-fs@freebsd.org Mon Aug 22 17:55:48 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B6C7EBC29F3 for ; Mon, 22 Aug 2016 17:55:48 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x236.google.com (mail-wm0-x236.google.com [IPv6:2a00:1450:400c:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4978A148F for ; Mon, 22 Aug 2016 17:55:48 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x236.google.com with SMTP id q128so131603556wma.1 for ; Mon, 22 Aug 2016 10:55:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=cdbt6AwK4UXZHgYVFVc27WqPDDxkXvGZDbCwA5UPfhY=; b=z+8+yHUCwNsYfuttfD3tF0voeWUSH2cVB7MahpgOM/jU2HynRVgvXWDnc36TvM3Ipo jQv4gpDNQWN1Xr0txmVhuuuFU4eXUeeBhZNXtqzIh6gJN2wcJZTen4z37VCz41a+c3cT 2WCZW6HGUdjnBvkFGOs7NMjSiRallTXj49b17TRw8WmfSpUaGA7AAYMCsj2ow+GJdNk+ 0nf8h/C7v/m/qeWx0kJqIfF6xcKRF+xOuCQ3Wh9Bhfnfst7cv4CvkRr/tIN+9bG/Xv+Q tfhm2ukPzoSZAPjLsD+fD5ne1EyD5iPLssb27G8XX6ABvZuC2VKNeEBJawMLAKdtiHZZ XM9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=cdbt6AwK4UXZHgYVFVc27WqPDDxkXvGZDbCwA5UPfhY=; b=KHiFjKKDxbmBezlPq29PIALKm6peYpkgxSKz4RaPy4FIyg/Cut1++19IimIGOyVyQa HBG3Y60meeojqoEuBX38OBvFfPFs0vcjCupr9jG2XM1s2lZkDlG6mVlIVAiE1+v3mQOq ECXEtmQpTXtDXr3lXxbU5EXKDTX6sCFa4robvLS7hjIIgdfEpZPfCqB3E0TLBlvq7KCp 98vbTcD01C9JtNHgMACUwOxVaI1ryo3EuRQPD1A/xFx2a62cp0AhhYpuJYRpq2+P5hKx mEzlKL0bchL84jK/mqlU0upGNUr0L1mSh+bHjz4Q/WIkantJ/h5/jp91I7RaiPJxCenK fVuA== X-Gm-Message-State: AEkoouu8Ne191f+QbzSREc697VF7eeRFno5O8+8dKl/kLW1Kv3zPyBDT+Y2koSiqPv2PqA== X-Received: by 10.28.19.134 with SMTP id 128mr15648784wmt.40.1471888546229; Mon, 22 Aug 2016 10:55:46 -0700 (PDT) Received: from macbook-air-de-benjamin-2.home (ANice-651-1-185-161.w83-197.abo.wanadoo.fr. [83.197.121.161]) by smtp.gmail.com with ESMTPSA id o2sm25430377wjo.3.2016.08.22.10.55.45 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 22 Aug 2016 10:55:45 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: Recommended HBA for ZFS, contemporary From: Ben RUBSON In-Reply-To: <059af3ed-71b3-b31d-893c-732121989cee@kateley.com> Date: Mon, 22 Aug 2016 19:55:44 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <633311CF-ADF2-479F-AFB9-E9265C101302@gmail.com> References: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> <059af3ed-71b3-b31d-893c-732121989cee@kateley.com> To: FreeBSD FS X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 17:55:48 -0000 > On 22 Aug 2016, at 19:13, Linda Kateley wrote: >=20 > Not just IT firmware, but you need to make sure the driver and = firmware match. >=20 > I try to use 16 if possible, 20 works but I have only found the = 20.00.04.00 version of firmware to be reliable. Would love to hear if = people have different experience. I am about to try the rocket raid in = IT mode because I am tired of fighting with lsi firmware. Very very = stable when you get the right combo, 1000's of phantom checksum errors = if not. FreeBSD 10.3 : # egrep ^mps0 /var/run/dmesg.boot mps0: mps0: Firmware: 20.00.04.00, Driver: 20.00.00.00-fbsd FreeBSD 11-RC1 : # egrep ^mps0 /var/run/dmesg.boot mps0: mps0: Firmware: 20.00.04.00, Driver: 21.01.00.00-fbsd Strangely enough there is no P21 driver/firmware @Avago : = http://www.avagotech.com/products/server-storage/host-bus-adapters/sas-921= 1-8i#downloads So I assume firmware P20 is the right one to use with FreeBSD 11. Well it is : = https://forums.freenas.org/index.php?threads/firmware-version-p21.45130/#p= ost-304399 I did not test firmwares 20.00.05.00 / 20.00.06.00 / 20.00.07.00. Did you ? Unfortunately changelog for 20.00.05.00 is no more available online, so = no idea what it does. The 2 others are, 3 + 1 defects corrected, should be worth it at first = glance. > I am starting to like 12g 3008 cards better as the firmware seems to = be a little more stable at version 9. Been testing 12 and so far so = good, but haven't run my larger stress tests yet.. >=20 > I would definitely go to the avago site and make sure there is driver = and firmware for the card you are purchasing before you buy. >=20 > Linda >=20 > On 8/22/16 11:18 AM, Ben RUBSON wrote: >>> On 22 Aug 2016, at 18:14, InterNetX - Juergen Gotteswinter = wrote: >>>=20 >>> LSI SAS 2008 Based HBA, there are dozens of OEM available for Budget = Prices >> With the IT firmware. >>=20 >>> Am 22.08.2016 um 18:08 schrieb Dmitry Morozovsky: >>>> Dear colleagues, >>>>=20 >>>> what is the list of preferred disk controllers to build storage = server, say >>>> 12-24 amd more 3.5" disks? Not too much IOPS are expected, and = some could be >>>> mitigated with L2ARC with only metadata cached, I suppose. >>>>=20 >>>> Two usage schemes are file server for big files (mostly cold = storage) and >>>> backup/surveillance server (constant stream writes, occasional = random reads) >>>>=20 >>>> Any hints? Thanks! >>>>=20 >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to = "freebsd-fs-unsubscribe@freebsd.org" >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Mon Aug 22 18:03:26 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58FBCBC2BD7 for ; Mon, 22 Aug 2016 18:03:26 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0F5AA1A4D for ; Mon, 22 Aug 2016 18:03:25 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id 41C70414E for ; Mon, 22 Aug 2016 13:03:23 -0500 (CDT) Subject: Re: Recommended HBA for ZFS, contemporary To: freebsd-fs@freebsd.org References: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> <059af3ed-71b3-b31d-893c-732121989cee@kateley.com> <633311CF-ADF2-479F-AFB9-E9265C101302@gmail.com> From: Karl Denninger Message-ID: Date: Mon, 22 Aug 2016 13:03:23 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <633311CF-ADF2-479F-AFB9-E9265C101302@gmail.com> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms010407070402010709060409" X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 18:03:26 -0000 This is a cryptographically signed message in MIME format. --------------ms010407070402010709060409 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 8/22/2016 12:55, Ben RUBSON wrote: >> On 22 Aug 2016, at 19:13, Linda Kateley wrote: >> >> Not just IT firmware, but you need to make sure the driver and firmwar= e match. >> >> I try to use 16 if possible, 20 works but I have only found the 20.00.= 04.00 version of firmware to be reliable. Would love to hear if people ha= ve different experience. I am about to try the rocket raid in IT mode bec= ause I am tired of fighting with lsi firmware. Very very stable when you = get the right combo, 1000's of phantom checksum errors if not. > FreeBSD 10.3 : > # egrep ^mps0 /var/run/dmesg.boot > mps0: > mps0: Firmware: 20.00.04.00, Driver: 20.00.00.00-fbsd > > FreeBSD 11-RC1 : > # egrep ^mps0 /var/run/dmesg.boot > mps0: > mps0: Firmware: 20.00.04.00, Driver: 21.01.00.00-fbsd > > Strangely enough there is no P21 driver/firmware @Avago : > http://www.avagotech.com/products/server-storage/host-bus-adapters/sas-= 9211-8i#downloads > So I assume firmware P20 is the right one to use with FreeBSD 11. > Well it is : https://forums.freenas.org/index.php?threads/firmware-vers= ion-p21.45130/#post-304399 > > I did not test firmwares 20.00.05.00 / 20.00.06.00 / 20.00.07.00. > Did you ? > Unfortunately changelog for 20.00.05.00 is no more available online, so= no idea what it does. > The 2 others are, 3 + 1 defects corrected, should be worth it at first = glance. > No problems here with: mps0: Firmware: 19.00.00.00, Driver: 21.00.00.00-fbsd This was on 10.x and is now on 11-PRE --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms010407070402010709060409 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjA4MjIxODAzMjNaME8GCSqGSIb3DQEJBDFCBEAB QH4g0slFDo0/U3+KbUKVLhx3pLBbA4E3zvIyxGF67xVnsqbCKD08vRRFvBT1C0XA0+oIj1RB ++Bf8ts+t27zMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIAoF8aIPYO 4jRWZPAWpUTYc7pHxAJp2/8pSsX0cCe4FwSkR0iXCFFkVTVsCWrKjftDQZ20zh7z/FnIT7Xg jxHq7BLQ6PgAGsRmO9sFZY50Fg2H3EP5toGFKMpF8kZu55k1DHdZbcsZR3O2oBCpiQjiYW81 OrQBajl3QS9Xd3CFAS8Sv+y8sziAJGrHXaNt7IIzGvVVPqbjBHCsLxmm43Jlu9Xe/L3Pmjqr rjBPdDOipMeSHtMzT6AVwHwzk/kEQCexvaw8zJFiZ9uPShLsaoWom9iwhKwV8JQ8F4FxTHDc kEn8X7mL+ijp+ytdhBO8n/YW8KEt5Twh8GXlX1yVVgHDVB6vKQa3hUT6M2bCYTCi6414CQEh gnnX3jVU44ajKg8dlfOdzCFTs5iTc3WN5Oz9gCOXJDfOrnHEqqXIyGX45mlJKJZx6/RvnUeJ ftfdzm5j7KZW/7NuoA6VVioXS11tzqO0vkdr+2RwMFE885jx3Ikwavn1+WJBo8h9+RY9Tbi+ 6EApnO9L8zh+0/7oGCaNKubO/nBHW0U2mnHmErPYYhYK76cGPhxu4EK8vWC80R28wg5syv0P XFZ0574AKOW6rXprXCeVfgQmDCzzJT7MCfgDmJQxzz9zrvYN2WqK2wkSSyl8vhayidOPcmp0 V7qoRsUrc02Tf3JEFsgM6/62zsUAAAAAAAA= --------------ms010407070402010709060409-- From owner-freebsd-fs@freebsd.org Mon Aug 22 18:34:55 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C03FDBC261F for ; Mon, 22 Aug 2016 18:34:55 +0000 (UTC) (envelope-from the.lists@mgm51.com) Received: from oneyou.mcmli.com (oneyou.mcmli.com [IPv6:2607:f2f8:af30::100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "oneyou.mcmli.com", Issuer "RapidSSL SHA256 CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A85371508 for ; Mon, 22 Aug 2016 18:34:55 +0000 (UTC) (envelope-from the.lists@mgm51.com) Received: from sentry.24cl.com (sentry.24cl.com [IPv6:2001:558:6017:94:4cfb:c574:24a0:7961]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "sentry.24cl.com", Issuer "Mike's Certificate Authority" (verified OK)) by oneyou.mcmli.com (Postfix) with ESMTPS id 3sJ2L65PNyz4D1q for ; Mon, 22 Aug 2016 14:34:54 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mgm51.com; s=mgm51-02; t=1471890894; bh=Lew0gV0iWxBkibT3t6xaRPEfAio4RtgmDGX/efdg46k=; h=Subject:To:From:Message-ID:Date; b=OpipX95BDsYegxM240Z0S3KNDUzwr+v3J+Sxnppp6GSbX11AJjk2a3iyRBHkyTZ5H PkBDTKfJ4X1KKnSee+et0IFBaL64z+8nD0ysbPYvo3pugNonwoTxRTAv2QDAnETHw9 7cePnjQ1Th87MWoW2REZ8go/IoC6THTfVVdZjcn/DHCpki8Qzz9WD7xTN3a3k3qmw8 NX386ZYDX2zI3g2L54OYttMz+bhnWRaFVBtg9G85kzmHu6IcSwswb2DrjTZgFNHqjT X4a1kF8AnLVgahTvs6Srqm9aGoHvy6AQcX5R5tp2oSFwXiA8gAh80wjYwFFHjRTCtJ efQwt8lv4Pmiw== Received: from [IPv6:fdcf:b715:2f4d:1:49a0:b0b7:349e:6605] (unknown [IPv6:fdcf:b715:2f4d:1:49a0:b0b7:349e:6605]) by sentry.24cl.com (Postfix) with ESMTP id 3sJ2L54MHpzGHHD for ; Mon, 22 Aug 2016 14:34:53 -0400 (EDT) Subject: Re: Recommended HBA for ZFS, contemporary To: freebsd-fs@freebsd.org References: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> From: Mike Message-ID: Date: Mon, 22 Aug 2016 14:34:36 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 18:34:55 -0000 On 8/22/2016 12:18 PM, Ben RUBSON wrote: > >> On 22 Aug 2016, at 18:14, InterNetX - Juergen Gotteswinter wrote: >> >> LSI SAS 2008 Based HBA, there are dozens of OEM available for Budget Prices > > With the IT firmware. What's the significance of "IT firmware"? From owner-freebsd-fs@freebsd.org Mon Aug 22 18:37:01 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A7432BC26D8 for ; Mon, 22 Aug 2016 18:37:01 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5D3A915F3 for ; Mon, 22 Aug 2016 18:37:01 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id 24686432F for ; Mon, 22 Aug 2016 13:36:59 -0500 (CDT) Subject: Re: Recommended HBA for ZFS, contemporary To: freebsd-fs@freebsd.org References: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> From: Karl Denninger Message-ID: Date: Mon, 22 Aug 2016 13:36:58 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms040103000101050401080704" X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 18:37:01 -0000 This is a cryptographically signed message in MIME format. --------------ms040103000101050401080704 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 8/22/2016 13:34, Mike wrote: > On 8/22/2016 12:18 PM, Ben RUBSON wrote: >>> On 22 Aug 2016, at 18:14, InterNetX - Juergen Gotteswinter wrote: >>> >>> LSI SAS 2008 Based HBA, there are dozens of OEM available for Budget = Prices >> With the IT firmware. > > What's the significance of "IT firmware"? > > It's (slightly) faster, provided you don't turn on any of the other stuff (which is always dumb with these adapters.) --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms040103000101050401080704 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjA4MjIxODM2NThaME8GCSqGSIb3DQEJBDFCBEDG 3IiPgINsQoMXwJIaJQLzOqo2emH2PmyKRpv8vEcZ0YGkGNUgyiOp2n1tt0bdQ5D9b953zgrA Flac9Bhr7N1AMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIABMa8kDL9 YnB6hBVJLFJ7fOsMZRNnO65nSVWj/sP9zVVvVVqo2t2c7hgbc95vrh5PcPhG5lL8x6//QrCr oLQXWHH/svdvhIrBnyOi/0xyPz/82/aOahcYQfhi6azoQfijeWpV8YxkWbdFtufWwu77lW9/ u4+6JrNDEUCS1NMyaOIkxuOIWi4R774ueMy/Rw1KbHYlj4O82rg7J2dDu/rnyu9Otg1RWBHO SBgcV/dg3YvfkHpvFFDOCkRLdlCL/kjonqDXPVUyx78S9wwIF1d2H+kbjM1EKFcGJOBYGzXG 1MbW3mkkxC/OffU+OnzjVKpN5/WtPjlmFYPpbdcGBC6ZOPQir4aLOhBGm63LUxy3EL+08LOM nCJrRM2stvuOERRyrxAnlrFRo+1OMtrtfi/ZH5EkhDmhQgxm8/SCHEscTEFC0Ie1cH0Gfi38 yd69YWXscWfHPpxfFovg/NY5mv/fjZ3Habs10k2T/MfqVR2H1MuRtRvuBp/IDx9Svpj7O/Li NT74wla77Ykt5Yd/rbBHy2d1YZukNP1uWF8sOubUB3fQlM73KfSHcL7WlOrPwS+MK5/f31W2 cNvgdzrZOifbfFh3REctyLldmbe1vukyuc6z0LCdakbrEwoXVQlZRszrVW2b1sbcu9KyNU5G 2gEmP1z/8CC5WGySLwf9G4JWsBkAAAAAAAA= --------------ms040103000101050401080704-- From owner-freebsd-fs@freebsd.org Mon Aug 22 18:37:04 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 505D9BC26F3 for ; Mon, 22 Aug 2016 18:37:04 +0000 (UTC) (envelope-from marck@rinet.ru) Received: from woozle.rinet.ru (woozle.rinet.ru [195.54.192.68]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D0D8315F7 for ; Mon, 22 Aug 2016 18:37:03 +0000 (UTC) (envelope-from marck@rinet.ru) Received: from localhost (localhost [127.0.0.1]) by woozle.rinet.ru (8.14.5/8.14.5) with ESMTP id u7MIaxQP085315; Mon, 22 Aug 2016 21:36:59 +0300 (MSK) (envelope-from marck@rinet.ru) Date: Mon, 22 Aug 2016 21:36:59 +0300 (MSK) From: Dmitry Morozovsky To: Mike cc: freebsd-fs@freebsd.org Subject: Re: Recommended HBA for ZFS, contemporary In-Reply-To: Message-ID: References: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-NCC-RegID: ru.rinet X-OpenPGP-Key-ID: 6B691B03 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (woozle.rinet.ru [0.0.0.0]); Mon, 22 Aug 2016 21:36:59 +0300 (MSK) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 18:37:04 -0000 On Mon, 22 Aug 2016, Mike wrote: > >> LSI SAS 2008 Based HBA, there are dozens of OEM available for Budget Prices > > > > With the IT firmware. > > What's the significance of "IT firmware"? ability to export raw disk device withou unnatural intelligence of a controller to mangle with data :) -- Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] [ FreeBSD committer: marck@FreeBSD.org ] ------------------------------------------------------------------------ *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru *** ------------------------------------------------------------------------ From owner-freebsd-fs@freebsd.org Mon Aug 22 18:38:54 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6CABCBC2779 for ; Mon, 22 Aug 2016 18:38:54 +0000 (UTC) (envelope-from the.lists@mgm51.com) Received: from oneyou.mcmli.com (oneyou.mcmli.com [IPv6:2607:f2f8:af30::100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "oneyou.mcmli.com", Issuer "RapidSSL SHA256 CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 535D41737 for ; Mon, 22 Aug 2016 18:38:54 +0000 (UTC) (envelope-from the.lists@mgm51.com) Received: from sentry.24cl.com (sentry.24cl.com [IPv6:2001:558:6017:94:4cfb:c574:24a0:7961]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "sentry.24cl.com", Issuer "Mike's Certificate Authority" (verified OK)) by oneyou.mcmli.com (Postfix) with ESMTPS id 3sJ2Qj75gVz4D28 for ; Mon, 22 Aug 2016 14:38:53 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mgm51.com; s=mgm51-02; t=1471891134; bh=ivabfTfHQUN876NqFS+rLDnPxlVQjd5dB38RY2gvCVM=; h=Subject:To:From:Message-ID:Date; b=d6Ec3ZpgOVRJtDLwU2yZTExeBYx2I7sz4z9hO9Pzs3pFq4gWy6/a4LDH26+iLnv+I EZHezrwe/Hy7qnD+sUmG7n0KTHhoJklpJryQhXmKs6twxG7Wfokx7pSSgKg3wEltCI bSrhRPbsxtqPGuZFh6JB+wkwOxtwj5Gi/TOp0objz08/Zd8hYj/qxw6spRnii0XrBx KkHd7+W2r0nDeqMqT4vSuo/R44XnkgrjemiI1pia+C4BEAKFl12u/kQJiEG5KJVXH8 5tJVb+R6t+oRurr3qUUSzZ6mXJR7KLyIP/4SV3a60QJQMi3zQ22C5aAbxgTwFU5xmm mrSNds+uE+2Og== Received: from [IPv6:fdcf:b715:2f4d:1:49a0:b0b7:349e:6605] (unknown [IPv6:fdcf:b715:2f4d:1:49a0:b0b7:349e:6605]) by sentry.24cl.com (Postfix) with ESMTP id 3sJ2Qh6XlNzGHHD for ; Mon, 22 Aug 2016 14:38:52 -0400 (EDT) Subject: Re: Recommended HBA for ZFS, contemporary To: freebsd-fs@freebsd.org References: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> From: Mike Message-ID: Date: Mon, 22 Aug 2016 14:38:35 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 18:38:54 -0000 On 8/22/2016 2:36 PM, Karl Denninger wrote: > On 8/22/2016 13:34, Mike wrote: >> On 8/22/2016 12:18 PM, Ben RUBSON wrote: >>>> On 22 Aug 2016, at 18:14, InterNetX - Juergen Gotteswinter wrote: >>>> >>>> LSI SAS 2008 Based HBA, there are dozens of OEM available for Budget Prices >>> With the IT firmware. >> >> What's the significance of "IT firmware"? >> >> > It's (slightly) faster, provided you don't turn on any of the other > stuff (which is always dumb with these adapters.) Does it come with the HBA board(s), needs to be purchased or downloaded separately? Total noob here who wants to move my ZFS server disks off the motherboard SATA ports. :) From owner-freebsd-fs@freebsd.org Mon Aug 22 18:45:27 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B4F79BC2A17 for ; Mon, 22 Aug 2016 18:45:27 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6F0A71D16 for ; Mon, 22 Aug 2016 18:45:26 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id B667043DA for ; Mon, 22 Aug 2016 13:45:25 -0500 (CDT) Subject: Re: Recommended HBA for ZFS, contemporary To: freebsd-fs@freebsd.org References: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> From: Karl Denninger Message-ID: Date: Mon, 22 Aug 2016 13:45:25 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms030600060000020006090608" X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 18:45:27 -0000 This is a cryptographically signed message in MIME format. --------------ms030600060000020006090608 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 8/22/2016 13:38, Mike wrote: > On 8/22/2016 2:36 PM, Karl Denninger wrote: >> On 8/22/2016 13:34, Mike wrote: >>> On 8/22/2016 12:18 PM, Ben RUBSON wrote: >>>>> On 22 Aug 2016, at 18:14, InterNetX - Juergen Gotteswinter wrote: >>>>> >>>>> LSI SAS 2008 Based HBA, there are dozens of OEM available for Budge= t Prices >>>> With the IT firmware. >>> What's the significance of "IT firmware"? >>> >>> >> It's (slightly) faster, provided you don't turn on any of the other >> stuff (which is always dumb with these adapters.) > Does it come with the HBA board(s), needs to be purchased or downloaded= > separately? I've never gotten a card with it on it; I've always had to flash it.=20 Flashing these cards is a bit of a highwire act and can brick them, but I've never had it happen myself. With that said my real reason to flash all of mine is to have all of them on the same firmware just so I can eliminate that as a possible contributor to problems if one starts acting up and the others are not. It's more of a "good practice" thing than a need IMHO; it's not a lot of fun to try to run down a problem only to find out after much hair has been pulled out of one's own head that the problem is bad firmware on a board! > Total noob here who wants to move my ZFS server disks off the > motherboard SATA ports. :) > --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms030600060000020006090608 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjA4MjIxODQ1MjVaME8GCSqGSIb3DQEJBDFCBEBZ 5yEUOJ02dP5E6xb+Juu5L7/w6e+VBhaYsxFfqj/MvFB3FJFyQLyEuixYpKD8plFbbwIZ6nAL Do/TWPgUkM70MGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIAjqx+ZgZy ms4oauWTDmlEqVdynFxNgjwmRfDRL5qVr16mrjlJdtuznhF9JdxX7GPqL+hrzp2s9VRKI6o4 uxhQ8QLMbwnmzKP/oBa1WhvPYDFia3pp5HaW/bhBy5FgVDA8ap/5EWwfjfEObgyGrs8YWFNI 2o6CwmQL5UEdcJhf6NGOYEIGjFLOpR8XMXHmT6LZj7C0mrEsqtmDX9KqL4p+30SVUTSdM7Qo zMjoU2dkoj2UAKsd/wgV8+wM7UKqCBFOcuAF9rsLfPDad3gV0MlUzWgHbzeJTbXs6X5KRNPZ Z5qdaoLLsqu55HKZoMlJ5FMszoJPp8OGBoC6X4ZwIVG3rSsCvzXn42ESGy+rkbdYPf2OYp4h L3Ow1S3ZO8ezG0BU+XzsGMVV9G2KcCFUCOLinFJbIA+gcZRk7CelgGdjKsuYavZFOM2k4iow 6OO2QpKw6CUutXOX29b8t2HpAwdxiTprdAy9Wkoun08BwpDwDU/5hBLFo0D7IUDs4hBuRnCS Je4o7j446uBtOZcq3Cqw8L1lfbHOsYDpL3qBzy4vUxjEYhSfWeUw33nnKJhDROeMR5Eoi3TM EQ2h/0ARHAQGyIXp82J82a2z8at4qPp7IL4NdZ2LmJWJtcEQOPNciPK6KFQPjnlczdCYi8M6 A93wMLAn7b5qhgzqvR255kHcSAAAAAAAAAA= --------------ms030600060000020006090608-- From owner-freebsd-fs@freebsd.org Mon Aug 22 18:51:25 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4FB49BC2C42 for ; Mon, 22 Aug 2016 18:51:25 +0000 (UTC) (envelope-from spork@bway.net) Received: from smtp2.bway.net (smtp2.v6.bway.net [IPv6:2607:d300:1::28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1295D11BB for ; Mon, 22 Aug 2016 18:51:25 +0000 (UTC) (envelope-from spork@bway.net) Received: from frankentosh.sporklab.com (pool-173-70-218-98.nwrknj.fios.verizon.net [173.70.218.98]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: spork@bway.net) by smtp2.bway.net (Postfix) with ESMTPSA id 7401195878; Mon, 22 Aug 2016 14:51:13 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bway.net; s=mail; t=1471891873; bh=6x0Wqj5yT5OxP2gMIbrzdO2nqIXjwVW7dCAJsNPOSrM=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=DIpnC5R6juEMpBZkK5AIFQF1VYAVevjAtyKJ8x2QcSlhN/voATlflSMxmTZebw+RU OEKkUXtTNc+hRqPQtsum75KhE3AFjlsuj3C7sOcSZ1rTascPF/f7EReaD7DVKveoi0 lTXFFZjPlvwvxB/G2s9FBlrdk2fTAI5JZKeYQobs= Subject: Re: Recommended HBA for ZFS, contemporary Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: multipart/signed; boundary="Apple-Mail=_076AD745-C99D-4C7B-A4ED-6398EE494B89"; protocol="application/pgp-signature"; micalg=pgp-sha256 X-Pgp-Agent: GPGMail From: Charles Sprickman In-Reply-To: Date: Mon, 22 Aug 2016 14:51:12 -0400 Cc: freebsd-fs@freebsd.org Message-Id: <63B2949F-01C6-4465-A2D8-E4AA0FE0150C@bway.net> References: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> To: Karl Denninger X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 18:51:25 -0000 --Apple-Mail=_076AD745-C99D-4C7B-A4ED-6398EE494B89 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 > On Aug 22, 2016, at 2:45 PM, Karl Denninger = wrote: >=20 > On 8/22/2016 13:38, Mike wrote: >> On 8/22/2016 2:36 PM, Karl Denninger wrote: >>> On 8/22/2016 13:34, Mike wrote: >>>> On 8/22/2016 12:18 PM, Ben RUBSON wrote: >>>>>> On 22 Aug 2016, at 18:14, InterNetX - Juergen Gotteswinter = wrote: >>>>>>=20 >>>>>> LSI SAS 2008 Based HBA, there are dozens of OEM available for = Budget Prices >>>>> With the IT firmware. >>>> What's the significance of "IT firmware"? >>>>=20 >>>>=20 >>> It's (slightly) faster, provided you don't turn on any of the other >>> stuff (which is always dumb with these adapters.) >> Does it come with the HBA board(s), needs to be purchased or = downloaded >> separately? > I've never gotten a card with it on it; I've always had to flash it. > Flashing these cards is a bit of a highwire act and can brick them, = but > I've never had it happen myself. With that said my real reason to = flash > all of mine is to have all of them on the same firmware just so I can > eliminate that as a possible contributor to problems if one starts > acting up and the others are not. It's more of a "good practice" = thing > than a need IMHO; it's not a lot of fun to try to run down a problem > only to find out after much hair has been pulled out of one's own head > that the problem is bad firmware on a board! >> Total noob here who wants to move my ZFS server disks off the >> motherboard SATA ports. :) >>=20 This is all a bit crazy. Is the LSI series of HBAs really the only = choice outside of onboard controllers? It seems like they are troublesome, = support is spotty, and this whole flashing process gives me nightmares. Also personally, with the few boards I=92ve seen with built-in LSI = adapters have always been a little wonky compared to their non-LSI counterparts. Is this all just the end-result of LSI basically buying up everyone? Charles >=20 > -- > Karl Denninger > karl@denninger.net > /The Market Ticker/ > /[S/MIME encrypted email preferred]/ --Apple-Mail=_076AD745-C99D-4C7B-A4ED-6398EE494B89 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBCAAGBQJXu0mhAAoJEMfwH0dqLIp2mqcH/3EJLcFBcmmzfPamY6NLuimp Esl+l1DLWqkmo/cHgo8rurYEdJiYu9W+3Hx4yRVsDBnJt60FGilSs2yUUGJevEKt sUvSkRdV6JS9ZIMpO04bQxDpo7d5IjvOFp6zQfnzlTY5u+NVT1ITm3/VRh3UXS6J jJSPk8QD9kBJJaS9lWWREuRQtwr47xFYqUYv06IdRiqTdQxo9LxOu/sbZ8Bds7QT J/QY6r2DmMjMU+XKRjhmK/l/AYPLVf0zX7ktxcuLlVDntRcMccfIEvgRkNyJFXH0 Yz03MgEHcnOyjLBr2FW7CEZDFb73zxQq/Bq2c53xDjAe7cHaD89gTS27yR0PnTk= =dnfo -----END PGP SIGNATURE----- --Apple-Mail=_076AD745-C99D-4C7B-A4ED-6398EE494B89-- From owner-freebsd-fs@freebsd.org Mon Aug 22 18:56:13 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6D43BBC2D76 for ; Mon, 22 Aug 2016 18:56:13 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 265D515C3 for ; Mon, 22 Aug 2016 18:56:12 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id BD3CC4472 for ; Mon, 22 Aug 2016 13:56:11 -0500 (CDT) Subject: Re: Recommended HBA for ZFS, contemporary References: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> <63B2949F-01C6-4465-A2D8-E4AA0FE0150C@bway.net> To: freebsd-fs@freebsd.org From: Karl Denninger Message-ID: <2d444b5e-be3a-b7d4-4bf3-f23e48005bb9@denninger.net> Date: Mon, 22 Aug 2016 13:56:10 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <63B2949F-01C6-4465-A2D8-E4AA0FE0150C@bway.net> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms010709050304000204060603" X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 18:56:13 -0000 This is a cryptographically signed message in MIME format. --------------ms010709050304000204060603 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 8/22/2016 13:51, Charles Sprickman wrote: >> On Aug 22, 2016, at 2:45 PM, Karl Denninger wrote= : >> >> On 8/22/2016 13:38, Mike wrote: >>> On 8/22/2016 2:36 PM, Karl Denninger wrote: >>>> On 8/22/2016 13:34, Mike wrote: >>>>> On 8/22/2016 12:18 PM, Ben RUBSON wrote: >>>>>>> On 22 Aug 2016, at 18:14, InterNetX - Juergen Gotteswinter wrote: >>>>>>> >>>>>>> LSI SAS 2008 Based HBA, there are dozens of OEM available for Bud= get Prices >>>>>> With the IT firmware. >>>>> What's the significance of "IT firmware"? >>>>> >>>>> >>>> It's (slightly) faster, provided you don't turn on any of the other >>>> stuff (which is always dumb with these adapters.) >>> Does it come with the HBA board(s), needs to be purchased or download= ed >>> separately? >> I've never gotten a card with it on it; I've always had to flash it. >> Flashing these cards is a bit of a highwire act and can brick them, bu= t >> I've never had it happen myself. With that said my real reason to fla= sh >> all of mine is to have all of them on the same firmware just so I can >> eliminate that as a possible contributor to problems if one starts >> acting up and the others are not. It's more of a "good practice" thin= g >> than a need IMHO; it's not a lot of fun to try to run down a problem >> only to find out after much hair has been pulled out of one's own head= >> that the problem is bad firmware on a board! >>> Total noob here who wants to move my ZFS server disks off the >>> motherboard SATA ports. :) >>> > This is all a bit crazy. Is the LSI series of HBAs really the only cho= ice > outside of onboard controllers? It seems like they are troublesome, su= pport > is spotty, and this whole flashing process gives me nightmares. They're not troublesome and I've got a LOT of them in production service with zero field failures. The flashing thing is a one-time deal; the IT firmware isn't really required, but it will keep you from considering doing something stupid at some point in the future (since the "stupid" features aren't in the IT firmware!) > Also personally, with the few boards I=92ve seen with built-in LSI adap= ters > have always been a little wonky compared to their non-LSI counterparts.= > > Is this all just the end-result of LSI basically buying up everyone? > > Charles These are one of those "they made a good one, and nobody's made a better one" deals. I have used other adapters (Adaptec, the PERC series, etc) but they have all sucked to some degree. The LSIs have been the one HBA that has never made me want to do something evil to their engineers. Just do it, get the right firmware on the card (once) and be happy. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms010709050304000204060603 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjA4MjIxODU2MTBaME8GCSqGSIb3DQEJBDFCBEBI +fIzqwaw8V0XEFHYRFTMNvyiXelPcoyf3vgVVQsrFrhNV75PkgruoYEbnwm1pmNN9+kmT34y VxKER6hyFNwwMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIArzgAAm9U ygAZ4sVQwoGXF8UnjtIzlzli+iXRVYodRtrm6u9OkAuNro63dcRq2vxhLW0dHSCIH5derAZI eNF/livHeuIRzwdipbbtfXlyFLpVTb30mA8N/lzP8nKmh+mLl+uDZ7hbhw4MtUpTfd1udbK/ +ROZOSUZxvJcVd9/01GyIuaz4fTW8912qaS4E+xcLqQH6wiWAfrTDh0QOYUvc/nWKE+HM5Um M+aEfiOcU42GWWHWk2OaYbCo2uZ41ESbjqFcvyHc/S3rNtvyHANP51fOsWPFE8dRwXhuqUvq U+D6+aDKWt9ul7TMlaVW4OgaygIuy3jKjTL2twjQdd3z2oZ5U+syavnFEuPgXxzVkE2zlgeO v79AhfBGt9v+EatWal8bD5iZnKg5uJqyrNs79By4/xaWMAoq3Iop3VTgBcDFUlrpyDD+ro60 5qVmc9Lm0KzmC/tOB88Zf9QyLvY69qAxOvdri+rvn6+4UyeAvvrhYQ7Hq9aFGEZ1+MuDSTgs I67KL3kyw7OnRRw9H9HGtoCwt9WiURg0iHcDFZewRFYXxD9KfbBkZI7Zf6vm3Xtx7T4Glwlc PbZRRsg+JBL5kJhrsaw+0QKXCvO/uZAUy0l5Nk62AMrsdJ8NUjNT7vYuJo6vxjmrvBXadL4x B+8xR/J1wAvXHgF0Ta7J6qZd3NEAAAAAAAA= --------------ms010709050304000204060603-- From owner-freebsd-fs@freebsd.org Mon Aug 22 19:01:39 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EF6C4BC2EE2 for ; Mon, 22 Aug 2016 19:01:39 +0000 (UTC) (envelope-from lkateley@kateley.com) Received: from mail-it0-x232.google.com (mail-it0-x232.google.com [IPv6:2607:f8b0:4001:c0b::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BC7CA19C4 for ; Mon, 22 Aug 2016 19:01:39 +0000 (UTC) (envelope-from lkateley@kateley.com) Received: by mail-it0-x232.google.com with SMTP id x131so104110081ite.0 for ; Mon, 22 Aug 2016 12:01:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kateley-com.20150623.gappssmtp.com; s=20150623; h=reply-to:subject:references:to:from:organization:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=GvgBlFiD+wCetFqmP/rJXAVMuAd3QDNdalLSksyLm5E=; b=VIIZo9A5DJJA00qtSR/aY1Q0W6HPreUm7SzQi3cbN07YC7teZQw+UIwa5FWI8Uekfp bxKjozs5AHsTlWUE2doD0SBFjrvWAsDBImK0wgDhmzV7DgSVk18c9OPqX7swDehVytyd ElUWMPCHblbfTNzGGasCN2muVQJUOC7UuSwuiEuex8hdGjaJiKT1dcNRaQEH6gfkgQeU HB1NVlXw29FfZzaaItoHmeVEWUwMeONJlSsRxq2ugyqNyxQjy477Kczch/Dbr3D0FYTG F9dkEhuHr41bDkSqR1fObyOshjQ4rqH84Fqdc1UfVf7S2SCNgiU6OhFQ+yd6LIAfqVNT +wHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:reply-to:subject:references:to:from:organization :message-id:date:user-agent:mime-version:in-reply-to :content-transfer-encoding; bh=GvgBlFiD+wCetFqmP/rJXAVMuAd3QDNdalLSksyLm5E=; b=fdEavbLk/lhFWv6d3IUeIe3JsTTgPCSGdSxl8aWwab6rqXMCV3wYj90PeouC+8Eb4U DQOu5lMY5E0RhujJmLY0C5gXoUjkTJwJ0A6Ks7VY2yK+2sWcbbUno9llhFNd/wHX6c7B lJIuSrCoAFkIw6eBMwsWryXrJ5/KWUcWvv8xLS8wZHiBC2AGkSfRlqM+QPC+Hg3kafoR SOOqIY0Wlw/GxfvkXR4Hnucofaqj6rPVpXI8qXDB9taRDqzKwgibWlrjuNWQ1xA/0FfV Bew2iYNSXXZD3XhH4pk4LD75vXFI0KSaMthNRQ8PAkXTqmmVgOeTBv0Ueb/8LWB+rF4M cA3Q== X-Gm-Message-State: AEkoouvVuu6YNpaLx103XCgpTtUvLypOaADAz14e7NI2W0RqmZOKINEXrTL9sv25Y6xOOw== X-Received: by 10.36.46.203 with SMTP id i194mr21458940ita.69.1471892498624; Mon, 22 Aug 2016 12:01:38 -0700 (PDT) Received: from Kateleyco-iMac.local (c-50-188-36-30.hsd1.mn.comcast.net. [50.188.36.30]) by smtp.googlemail.com with ESMTPSA id r188sm8262026ith.7.2016.08.22.12.01.37 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Aug 2016 12:01:37 -0700 (PDT) Reply-To: linda@kateley.com Subject: Re: Recommended HBA for ZFS, contemporary References: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> <059af3ed-71b3-b31d-893c-732121989cee@kateley.com> <633311CF-ADF2-479F-AFB9-E9265C101302@gmail.com> To: freebsd-fs@freebsd.org From: Linda Kateley Organization: Kateley Company Message-ID: <0b986049-719b-715a-f8c4-cc3ee845c577@kateley.com> Date: Mon, 22 Aug 2016 14:01:36 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <633311CF-ADF2-479F-AFB9-E9265C101302@gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 19:01:40 -0000 On 8/22/16 12:55 PM, Ben RUBSON wrote: >> On 22 Aug 2016, at 19:13, Linda Kateley wrote: >> >> Not just IT firmware, but you need to make sure the driver and firmware match. >> >> I try to use 16 if possible, 20 works but I have only found the 20.00.04.00 version of firmware to be reliable. Would love to hear if people have different experience. I am about to try the rocket raid in IT mode because I am tired of fighting with lsi firmware. Very very stable when you get the right combo, 1000's of phantom checksum errors if not. > FreeBSD 10.3 : > # egrep ^mps0 /var/run/dmesg.boot > mps0: > mps0: Firmware: 20.00.04.00, Driver: 20.00.00.00-fbsd > > FreeBSD 11-RC1 : > # egrep ^mps0 /var/run/dmesg.boot > mps0: > mps0: Firmware: 20.00.04.00, Driver: 21.01.00.00-fbsd > > Strangely enough there is no P21 driver/firmware @Avago : > http://www.avagotech.com/products/server-storage/host-bus-adapters/sas-9211-8i#downloads > So I assume firmware P20 is the right one to use with FreeBSD 11. > Well it is : https://forums.freenas.org/index.php?threads/firmware-version-p21.45130/#post-304399 I have worked with those guys, and they consistently say there is no difference in the firmware revs, but... I have seen major differences.. The 21 sounds like an oem release. Not sure how you can putback and require oem driver on an opensource release. I have a bunch of systems that I just downrev'ed the freenas, or didn't upgrade. If there is a 21 oem driver and if you are running supermicro hardware you should be able to get the driver/firmware from them. Avago has the drivers and firmware for each release. Just want to make sure they are the same. Would love to leave everything on 16 for 6gb and 9 for 12gb(actually I really like the version 10 on 12g sas but avago pulled it). Both those are tried and true.. If you have a 9211-8e or i that's working, life is good. The other thing I have tested is running the firmware for i(internal) card on an e(external) card and seems to have no impact. Also have taken the firmware for the 9300 card and put in on a roc and it worked just fine, then the next system I work on it doesn't even go in.. > > I did not test firmwares 20.00.05.00 / 20.00.06.00 / 20.00.07.00. > Did you ? Yes I tested .07. I did well with it, but I have had others tell me they got the phantom checksum errors so I will rerun my tests.. I am actually working on a system right now that is showing phantom checksum errors with the 07 release, so hoping to downrev.. I really can't wait to start testing something else besides lsi, after 20 years hoping to find something new that works, actually almost everything else I have worked with works, but there is always some caveat .... I have a quanta coming next week with a rocket raid. I have friends in eu that swear by the quanta stuff and I work with a company that says they have 100's of them deployed, so it's worth a try..I am also going back to retesting some perc's. > Unfortunately changelog for 20.00.05.00 is no more available online, so no idea what it does. > The 2 others are, 3 + 1 defects corrected, should be worth it at first glance. It is really frustrating when trying to deploy more than one system. I keep copies of most of the older firmware just in case. The documentation for avago firmware is horrific. The stuff that works well is the old stuff the 9211's and 9300's. The newer stuff like roc's are very hard to get firmware for unless you find the handful of people who know at supermicro or lsi. When you get into oem, then you have to trust your vendor. There is a about 30 or so storage vendors that i know of(so there is probably a bunch more) who have just taken freebsd or freenas or omni and put it onto some supermicro hardware and are selling as a storage product. There are only a few I have run into who have created value-add. > >> I am starting to like 12g 3008 cards better as the firmware seems to be a little more stable at version 9. Been testing 12 and so far so good, but haven't run my larger stress tests yet.. >> >> I would definitely go to the avago site and make sure there is driver and firmware for the card you are purchasing before you buy. >> >> Linda >> >> On 8/22/16 11:18 AM, Ben RUBSON wrote: >>>> On 22 Aug 2016, at 18:14, InterNetX - Juergen Gotteswinter wrote: >>>> >>>> LSI SAS 2008 Based HBA, there are dozens of OEM available for Budget Prices >>> With the IT firmware. >>> >>>> Am 22.08.2016 um 18:08 schrieb Dmitry Morozovsky: >>>>> Dear colleagues, >>>>> >>>>> what is the list of preferred disk controllers to build storage server, say >>>>> 12-24 amd more 3.5" disks? Not too much IOPS are expected, and some could be >>>>> mitigated with L2ARC with only metadata cached, I suppose. >>>>> >>>>> Two usage schemes are file server for big files (mostly cold storage) and >>>>> backup/surveillance server (constant stream writes, occasional random reads) >>>>> >>>>> Any hints? Thanks! >>>>> >>>> _______________________________________________ >>>> freebsd-fs@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Mon Aug 22 19:07:25 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C7EAEBC2216 for ; Mon, 22 Aug 2016 19:07:25 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-it0-x232.google.com (mail-it0-x232.google.com [IPv6:2607:f8b0:4001:c0b::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9C6441149 for ; Mon, 22 Aug 2016 19:07:25 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-it0-x232.google.com with SMTP id n128so96239665ith.1 for ; Mon, 22 Aug 2016 12:07:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=vzKDN/ITsObG2sNQh+JB0Cq4aaYVA1L8NvLi8sH+mM4=; b=ELZF9jOhAofPycR8QrRZC5+7Jq10WFjaMMrdnLJkSpQXEALzLiSSdB4qSAiJkt32Eo K7xcLAu16rrKR7ZoZggT3z/9yHs3L02BH2vT46GQfsudrL328KMyHo/Y13wtKPDxzzGP OMVVDO3aV/ni1/im4fB4+/9MjGK7RSrYJrRHM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=vzKDN/ITsObG2sNQh+JB0Cq4aaYVA1L8NvLi8sH+mM4=; b=Uv0LO2Ic8JxVzuPxJxKFPVstVV59RbVriifP+ETntj0On7iESPu3MCC8jL5yrXR25O 9rUaCWiW5y8Ricv2ThBD4nXLCHkBlr8U8akAeg/KSgj2fkFDctEyYARo4eivuYvFt9g1 yJfQkXVa8FsVpNZm+rlqjNkMXKaj/Hb76eqUbHjjCOhf8s+irzA7QbG60eAalnxk+osW aOiOsEJjKFbvjkKHdW2vSUyOuprAmwVF02CVCvA3Fp+k5MGcoak2xiJBuhQmzNlC0SpZ DjamFxMHZPzJq97+iBUmzzmKf19YgrMym9TTIbgXJzEdk41qk9H0QMUoW4xok7mj4H0R RZOw== X-Gm-Message-State: AEkoouu1WIB/IaQ7Osz+0C5uHlJMGwSym6T7e/oBp3VGJ6GhIfsgQ+BtONE6zlRqyCbYlxEWcoMorMeYPo7bcIYX X-Received: by 10.107.41.67 with SMTP id p64mr24316150iop.130.1471892845008; Mon, 22 Aug 2016 12:07:25 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.9.65 with HTTP; Mon, 22 Aug 2016 12:07:24 -0700 (PDT) From: Matthew Ahrens Date: Mon, 22 Aug 2016 12:07:24 -0700 Message-ID: Subject: OpenZFS Developer Summit 2016 - speakers announced, register today! To: developer , "developer@lists.illumos.org" , illumos-zfs , zfs-devel@freebsd.org, zfs-devel@zfsonlinux.org, freebsd-fs , zfs-discuss , "admin@open-zfs.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 19:07:25 -0000 I'm pleased to announce a great lineup of 9 talks at the 2016 OpenZFS Developer Summit, which will be held September 26-27th in San Francisco: State of the Union - Matt Ahrens of Delphix Keynote - Dustin Kirkland of Canonical ZFS-Native Encryption - Tom Caputi of Datto Scrub/Resilver Performance - Saso Kiselkov of Nexenta Fault Management - Don Brady of Intel ZFS Validation & QA - Sydney Vanda & John Salinas of Intel Lustre, Supercomputers, and ZFS - Brian Behlendorf of LLNL ZFS and Containers - Michael Crogan Noms Database - Adam Leventhal of Sutter Hill Ventures We'll also have an hour of presentations on the 2nd day, the morning of the hackathon. These will be 5-minute lightning updates on projects which have been presented at previous conferences. For example, we'll have an exciting announcement about Channel Programs, which was presented at our first conference in 2013. If you are interested in giving one of these updates, please contact me. Registration is open and space is limited, so please register today! https://www.eventbrite.com/e/openzfs-developer-summit-2016-tickets-24968191533%7COpenZFS All the details, including a list of the lightning talks, are available on the website: http://open-zfs.org/wiki/OpenZFS_Developer_Summit --matt p.s. There are also sponsorship opportunities still available. Contact me if your company is interested in sponsoring the conference. From owner-freebsd-fs@freebsd.org Mon Aug 22 19:09:07 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E03E1BC2370 for ; Mon, 22 Aug 2016 19:09:07 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from mail.in-addr.com (mail.in-addr.com [IPv6:2a01:4f8:191:61e8::2525:2525]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AB23413D1 for ; Mon, 22 Aug 2016 19:09:07 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from gjp by mail.in-addr.com with local (Exim 4.87 (FreeBSD)) (envelope-from ) id 1bbuan-0005hZ-0w; Mon, 22 Aug 2016 20:09:05 +0100 Date: Mon, 22 Aug 2016 20:09:05 +0100 From: Gary Palmer To: Linda Kateley Cc: freebsd-fs@freebsd.org Subject: Re: Recommended HBA for ZFS, contemporary Message-ID: <20160822190904.GA56297@in-addr.com> References: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> <059af3ed-71b3-b31d-893c-732121989cee@kateley.com> <633311CF-ADF2-479F-AFB9-E9265C101302@gmail.com> <0b986049-719b-715a-f8c4-cc3ee845c577@kateley.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <0b986049-719b-715a-f8c4-cc3ee845c577@kateley.com> X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on mail.in-addr.com); SAEximRunCond expanded to false X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 19:09:08 -0000 On Mon, Aug 22, 2016 at 02:01:36PM -0500, Linda Kateley wrote: > > > On 8/22/16 12:55 PM, Ben RUBSON wrote: > >> On 22 Aug 2016, at 19:13, Linda Kateley wrote: > >> > >> Not just IT firmware, but you need to make sure the driver and firmware match. > >> > >> I try to use 16 if possible, 20 works but I have only found the 20.00.04.00 version of firmware to be reliable. Would love to hear if people have different experience. I am about to try the rocket raid in IT mode because I am tired of fighting with lsi firmware. Very very stable when you get the right combo, 1000's of phantom checksum errors if not. > > FreeBSD 10.3 : > > # egrep ^mps0 /var/run/dmesg.boot > > mps0: > > mps0: Firmware: 20.00.04.00, Driver: 20.00.00.00-fbsd > > > > FreeBSD 11-RC1 : > > # egrep ^mps0 /var/run/dmesg.boot > > mps0: > > mps0: Firmware: 20.00.04.00, Driver: 21.01.00.00-fbsd > > > > Strangely enough there is no P21 driver/firmware @Avago : > > http://www.avagotech.com/products/server-storage/host-bus-adapters/sas-9211-8i#downloads > > So I assume firmware P20 is the right one to use with FreeBSD 11. > > Well it is : https://forums.freenas.org/index.php?threads/firmware-version-p21.45130/#post-304399 > I have worked with those guys, and they consistently say there is no > difference in the firmware revs, but... I have seen major differences.. > The 21 sounds like an oem release. Not sure how you can putback and > require oem driver on an opensource release. I have a bunch of systems > that I just downrev'ed the freenas, or didn't upgrade. If there is a 21 > oem driver and if you are running supermicro hardware you should be able > to get the driver/firmware from them. > > Avago has the drivers and firmware for each release. Just want to make > sure they are the same. Would love to leave everything on 16 for 6gb and > 9 for 12gb(actually I really like the version 10 on 12g sas but avago > pulled it). Both those are tried and true.. > > If you have a 9211-8e or i that's working, life is good. The other thing > I have tested is running the firmware for i(internal) card on an > e(external) card and seems to have no impact. Also have taken the > firmware for the 9300 card and put in on a roc and it worked just fine, > then the next system I work on it doesn't even go in.. > > > > > I did not test firmwares 20.00.05.00 / 20.00.06.00 / 20.00.07.00. > > Did you ? > Yes I tested .07. I did well with it, but I have had others tell me they > got the phantom checksum errors so I will rerun my tests.. I am actually > working on a system right now that is showing phantom checksum errors > with the 07 release, so hoping to downrev.. > > I really can't wait to start testing something else besides lsi, after > 20 years hoping to find something new that works, actually almost > everything else I have worked with works, but there is always some > caveat .... > I have a quanta coming next week with a rocket raid. I have friends in > eu that swear by the quanta stuff and I work with a company that says > they have 100's of them deployed, so it's worth a try..I am also going > back to retesting some perc's. AFAIK most (if not all) of the Dell PERC cards that I've seen are some form of LSI chip with Dell branding, firmware and drivers. Regards, Gary > > Unfortunately changelog for 20.00.05.00 is no more available online, so no idea what it does. > > The 2 others are, 3 + 1 defects corrected, should be worth it at first glance. > It is really frustrating when trying to deploy more than one system. I > keep copies of most of the older firmware just in case. The > documentation for avago firmware is horrific. The stuff that works well > is the old stuff the 9211's and 9300's. The newer stuff like roc's are > very hard to get firmware for unless you find the handful of people who > know at supermicro or lsi. When you get into oem, then you have to trust > your vendor. There is a about 30 or so storage vendors that i know of(so > there is probably a bunch more) who have just taken freebsd or freenas > or omni and put it onto some supermicro hardware and are selling as a > storage product. There are only a few I have run into who have created > value-add. > > > >> I am starting to like 12g 3008 cards better as the firmware seems to be a little more stable at version 9. Been testing 12 and so far so good, but haven't run my larger stress tests yet.. > >> > >> I would definitely go to the avago site and make sure there is driver and firmware for the card you are purchasing before you buy. > >> > >> Linda > >> > >> On 8/22/16 11:18 AM, Ben RUBSON wrote: > >>>> On 22 Aug 2016, at 18:14, InterNetX - Juergen Gotteswinter wrote: > >>>> > >>>> LSI SAS 2008 Based HBA, there are dozens of OEM available for Budget Prices > >>> With the IT firmware. > >>> > >>>> Am 22.08.2016 um 18:08 schrieb Dmitry Morozovsky: > >>>>> Dear colleagues, > >>>>> > >>>>> what is the list of preferred disk controllers to build storage server, say > >>>>> 12-24 amd more 3.5" disks? Not too much IOPS are expected, and some could be > >>>>> mitigated with L2ARC with only metadata cached, I suppose. > >>>>> > >>>>> Two usage schemes are file server for big files (mostly cold storage) and > >>>>> backup/surveillance server (constant stream writes, occasional random reads) > >>>>> > >>>>> Any hints? Thanks! > >>>>> > >>>> _______________________________________________ > >>>> freebsd-fs@freebsd.org mailing list > >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs > >>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > >>> _______________________________________________ > >>> freebsd-fs@freebsd.org mailing list > >>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs > >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > >> _______________________________________________ > >> freebsd-fs@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs > >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Mon Aug 22 19:16:40 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BCE93BC2570 for ; Mon, 22 Aug 2016 19:16:40 +0000 (UTC) (envelope-from lkateley@kateley.com) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F2601847 for ; Mon, 22 Aug 2016 19:16:40 +0000 (UTC) (envelope-from lkateley@kateley.com) Received: by mail-it0-x236.google.com with SMTP id x131so104971349ite.0 for ; Mon, 22 Aug 2016 12:16:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kateley-com.20150623.gappssmtp.com; s=20150623; h=reply-to:subject:references:to:from:organization:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=94tqfnnRKgDVHihBVevkYeM2t5WWH1t8di66q9RYexQ=; b=fD278Jgg3hiXQDV2GP/Do9goV6O6BbhS83H1uiNuyBhZi2DoKMIVNxdbdCHrLuyluv 0UA0YwrRMIaSXDGc1LlrKQzcLUoPKy4wXd3J2Af2eW+SqpMEQoAjbdlSwWoudc9WfMO/ NhvMkfqj4auABir1ITtgTY+X+W/5LFeqqS8jYEWOWg0J0wYbKqHjdEmhgXJSAwRH9dss wmcUBSDjerp34aVYOTLTrNzci5uqsWW5YeFeIAmgvZnOPuIRiO0BlrspGRjDusc+Dp/m l02XQGEtsPDAYIljmVa5rNziJA6hAng1Bk8yjPCCZBmDfe/gMk2yPEcXoTvMpPZk/8O5 uccw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:reply-to:subject:references:to:from:organization :message-id:date:user-agent:mime-version:in-reply-to :content-transfer-encoding; bh=94tqfnnRKgDVHihBVevkYeM2t5WWH1t8di66q9RYexQ=; b=P3L30QiFDO/fseV20GUSkxCsaueeEw+nQR5XC8jcVNyrgKeNNTqAzb9wBq2CrOZ+mj 7rIGdlApidX8Hx6N1Me6h30JD8nQIBV6yFrCJ+Sk/6IRVh7nh2uVIWb5kilKknNiIfBn gH0o0yIMNvDSD79ly7/iRNMAaLbWt+4vPp8955eaOaezW2JJWDmGhR+hct7sgTZaJtB3 hS6VbkidbS9w1SxzKz3fu84JoJJ+1gOt2UQDrsRJt97+VjHHPqIWhP7VQ9ppk/d/Ee8f tuj+CbXoC2YEa+sVvkFwnVe+7bINn1DVg/acmQfiMHprqRYQri7//uYtxuhuDVn8cVCw Zqpw== X-Gm-Message-State: AEkoous4BiXO2YGR9G0EbCTpRkwti6KcrC0QNWYy1CKYHqeug9BdyoOMmWFrPlPDxTYaZA== X-Received: by 10.107.132.104 with SMTP id g101mr29156015iod.147.1471893399661; Mon, 22 Aug 2016 12:16:39 -0700 (PDT) Received: from Kateleyco-iMac.local (c-50-188-36-30.hsd1.mn.comcast.net. [50.188.36.30]) by smtp.googlemail.com with ESMTPSA id w12sm27699iow.29.2016.08.22.12.16.38 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Aug 2016 12:16:39 -0700 (PDT) Reply-To: linda@kateley.com Subject: Re: Recommended HBA for ZFS, contemporary References: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> To: freebsd-fs@freebsd.org From: Linda Kateley Organization: Kateley Company Message-ID: <6f78c6ba-3118-8390-3956-64f573db4d54@kateley.com> Date: Mon, 22 Aug 2016 14:16:38 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 19:16:40 -0000 Their cards can be run in 2 different modes. One that does raid on the cards, and the other just passes through the devices. That is IT mode. ZFS does better if you give it control of the device. By default it wants to do things like like send a cache flush request to the disks. If you have a controller card that doesn't understand the request, it can hang. Or you can turn off cache flushing. On 8/22/16 1:34 PM, Mike wrote: > On 8/22/2016 12:18 PM, Ben RUBSON wrote: >>> On 22 Aug 2016, at 18:14, InterNetX - Juergen Gotteswinter wrote: >>> >>> LSI SAS 2008 Based HBA, there are dozens of OEM available for Budget Prices >> With the IT firmware. > > What's the significance of "IT firmware"? > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Mon Aug 22 19:23:47 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 314FEBC288C for ; Mon, 22 Aug 2016 19:23:47 +0000 (UTC) (envelope-from lkateley@kateley.com) Received: from mail-it0-x235.google.com (mail-it0-x235.google.com [IPv6:2607:f8b0:4001:c0b::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E854B1D3F for ; Mon, 22 Aug 2016 19:23:46 +0000 (UTC) (envelope-from lkateley@kateley.com) Received: by mail-it0-x235.google.com with SMTP id f6so96833559ith.0 for ; Mon, 22 Aug 2016 12:23:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kateley-com.20150623.gappssmtp.com; s=20150623; h=reply-to:subject:references:to:from:organization:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=6sVfVJYYnUKSczoMyiuIDDdciUxQlmZXzcjrWipZ2vs=; b=RARuV5LPlPT2RLFLhqaLMgYJF43qeKRodjFxw/ckn9GvRpJhlDSYuXRl47p7Pmp+rN eUzc3OZVu6EWmn4BN8ctuIlSwQxD/zQZ+Cas4u8hX67jeUfDUIiPNoMANXMctNEwmst3 1vvbrFicsoYyyDE5VPCAC/PC0oglt1UDtG5kVAN1BF1recacPeDgy0EFtsu+GwZm7wcq xoX8GgSN38Xc9Y/+PWoEUJ/phEe1iBIKPEw53wANDTvPULhb8Tf3WFY6PvetOLS18CjE Dy62hQo3ezGVirEeOtWQArAzH45df25GDM4DaPV93IaY5H/7FzKET2WsoUfiSGzb0iuW n9gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:reply-to:subject:references:to:from:organization :message-id:date:user-agent:mime-version:in-reply-to :content-transfer-encoding; bh=6sVfVJYYnUKSczoMyiuIDDdciUxQlmZXzcjrWipZ2vs=; b=KDqfa61kPdFBWc9Jd5402ZArm/2KDnFAqWl2+7TOT/d2VGAFDbgs/nqXgv0VzRaHLA ZLlJiaIJr5JUjmO1j7Kb3qtSjf/ptaDWa+oLFaQjFNVzA/27FzeAh7gK2BDUElp6rlqM 7WksytcyvfISG3n/eTVViHEHhJsFmepK7+1dt2z9EQynJbvr1CX3OuREMGhu48k28HQz 27aIPwc9S3b1oK4j8thDLE4gc9vn60GPGPkkRbgw0Sn3cxrjDoo16zKPFfZNgkPRixMp 4WweVWPS6f97C+YiknjyoENksOIZ/jUKUaquqWFqZvHYY36mdgZHBj1Ci1jQ1+efqvzM M0HA== X-Gm-Message-State: AEkooushhC99x2absA/Au5A7dXX/XqEzcm9MW8JEbJg/Uz5CjkaorBpDBl58a0v1bTVRuQ== X-Received: by 10.107.132.104 with SMTP id g101mr29192055iod.147.1471893826144; Mon, 22 Aug 2016 12:23:46 -0700 (PDT) Received: from Kateleyco-iMac.local (c-50-188-36-30.hsd1.mn.comcast.net. [50.188.36.30]) by smtp.googlemail.com with ESMTPSA id e6sm1439609ith.0.2016.08.22.12.23.45 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Aug 2016 12:23:45 -0700 (PDT) Reply-To: linda@kateley.com Subject: Re: Recommended HBA for ZFS, contemporary References: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> <63B2949F-01C6-4465-A2D8-E4AA0FE0150C@bway.net> To: freebsd-fs@freebsd.org From: Linda Kateley Organization: Kateley Company Message-ID: <3c17c513-2eb3-7acf-4885-b31594ea2ee8@kateley.com> Date: Mon, 22 Aug 2016 14:23:44 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <63B2949F-01C6-4465-A2D8-E4AA0FE0150C@bway.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 19:23:47 -0000 On 8/22/16 1:51 PM, Charles Sprickman via freebsd-fs wrote: >> On Aug 22, 2016, at 2:45 PM, Karl Denninger wrote: >> >> On 8/22/2016 13:38, Mike wrote: >>> On 8/22/2016 2:36 PM, Karl Denninger wrote: >>>> On 8/22/2016 13:34, Mike wrote: >>>>> On 8/22/2016 12:18 PM, Ben RUBSON wrote: >>>>>>> On 22 Aug 2016, at 18:14, InterNetX - Juergen Gotteswinter wrote: >>>>>>> >>>>>>> LSI SAS 2008 Based HBA, there are dozens of OEM available for Budget Prices >>>>>> With the IT firmware. >>>>> What's the significance of "IT firmware"? >>>>> >>>>> >>>> It's (slightly) faster, provided you don't turn on any of the other >>>> stuff (which is always dumb with these adapters.) >>> Does it come with the HBA board(s), needs to be purchased or downloaded >>> separately? >> I've never gotten a card with it on it; I've always had to flash it. >> Flashing these cards is a bit of a highwire act and can brick them, but >> I've never had it happen myself. With that said my real reason to flash >> all of mine is to have all of them on the same firmware just so I can >> eliminate that as a possible contributor to problems if one starts >> acting up and the others are not. It's more of a "good practice" thing >> than a need IMHO; it's not a lot of fun to try to run down a problem >> only to find out after much hair has been pulled out of one's own head >> that the problem is bad firmware on a board! >>> Total noob here who wants to move my ZFS server disks off the >>> motherboard SATA ports. :) >>> > This is all a bit crazy. Is the LSI series of HBAs really the only choice > outside of onboard controllers? It seems like they are troublesome, support > is spotty, and this whole flashing process gives me nightmares. > > Also personally, with the few boards I�ve seen with built-in LSI adapters > have always been a little wonky compared to their non-LSI counterparts. > > Is this all just the end-result of LSI basically buying up everyone? I have the same question. LSI used to always be the go-to, but in recent years it seems to get more and more wonky.. there has to be a better way. > > Charles > >> -- >> Karl Denninger >> karl@denninger.net >> /The Market Ticker/ >> /[S/MIME encrypted email preferred]/ From owner-freebsd-fs@freebsd.org Mon Aug 22 20:12:44 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E41A2BC267D for ; Mon, 22 Aug 2016 20:12:44 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x241.google.com (mail-wm0-x241.google.com [IPv6:2a00:1450:400c:c09::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 759021EA9 for ; Mon, 22 Aug 2016 20:12:44 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x241.google.com with SMTP id i138so15012683wmf.3 for ; Mon, 22 Aug 2016 13:12:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=uGtzlllpCRww/UjFy2cCP6kg9sleV1XDzRjft677g8A=; b=xJnq8Fv7NqMaAp4rf9hcqm0QudwHP9YzPuDFxX9cvMSbF+60dBaVAGPXHIuAc1ADJp 8KUjh9DfY/r2I1oIHyIb/bxNA7ycQk0P4hTpf0KVealKiv9WdTld6FWEHqNIlE46Q3rl 6lnX6YK+Dr+l36EZFgAEUZIBvkXCq62Z8+SXwPrjpaI/h36QktPo680iBLJ8TZYkdp3K hYXdm/i1G4JE2Vw7uQycsCywEn3MXd4MowZ0qWCf1e+RjFbtBdi9UvGCcGpmVvlF30yE cBVBpI6yL6WO2jakgYGKrHU2XYC4RbbLghqzrV/nyk/L80geyTG7jq2/DBxBfqukfH78 D87g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=uGtzlllpCRww/UjFy2cCP6kg9sleV1XDzRjft677g8A=; b=R+Twe0P8tRQMQZACnuE4915QqfP+5x3NocITATAzxpuVfJmDlmo5OO75KdfbcBW0PE RZuc793fL8PQBV6Deg+XDwD42yc0cJX4OeISjW1OKkBK5j7tixwguFo2xVtHQP2v2+G5 PwhCwobYdapf0LdB6TRehp6GKaBE8v5SjnsVJd0qBHnq80cJmg5i7TaBcFIv1UoNq41C WHC4bWvd/vnvLQ7XzBxreahhjIFuZkF5q3QvnJrMgSQKnB/r7kSx95v4ge2eYHjS19iS VHHB/bEkzFjKpxHenjsj8/zm5kkhHEAs77K31XdS+VgOrLKw3HMQmDEFxNturA7iEDeD 016g== X-Gm-Message-State: AEkoouvfSWM+5jB9NGUQbnbnI7w5ijTPg64XFpw0QnVdTwLIPn5fQmPTBPqk4XghDjoQnw== X-Received: by 10.194.221.6 with SMTP id qa6mr18436334wjc.191.1471896762474; Mon, 22 Aug 2016 13:12:42 -0700 (PDT) Received: from macbook-air-de-benjamin.home (ANice-651-1-185-161.w83-197.abo.wanadoo.fr. [83.197.121.161]) by smtp.gmail.com with ESMTPSA id bj8sm25975892wjc.49.2016.08.22.13.12.41 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 22 Aug 2016 13:12:41 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: Recommended HBA for ZFS, contemporary From: Ben RUBSON In-Reply-To: <0b986049-719b-715a-f8c4-cc3ee845c577@kateley.com> Date: Mon, 22 Aug 2016 22:12:42 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <3CC84BEA-EE4C-41E1-986E-2BF7A2784ADA@gmail.com> References: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> <059af3ed-71b3-b31d-893c-732121989cee@kateley.com> <633311CF-ADF2-479F-AFB9-E9265C101302@gmail.com> <0b986049-719b-715a-f8c4-cc3ee845c577@kateley.com> To: FreeBSD FS X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 20:12:45 -0000 > On 22 Aug 2016, at 21:01, Linda Kateley wrote: >=20 >=20 >=20 > On 8/22/16 12:55 PM, Ben RUBSON wrote: >>> On 22 Aug 2016, at 19:13, Linda Kateley = wrote: >>>=20 >>> Not just IT firmware, but you need to make sure the driver and = firmware match. >>>=20 >>> I try to use 16 if possible, 20 works but I have only found the = 20.00.04.00 version of firmware to be reliable. Would love to hear if = people have different experience. I am about to try the rocket raid in = IT mode because I am tired of fighting with lsi firmware. Very very = stable when you get the right combo, 1000's of phantom checksum errors = if not. >> FreeBSD 10.3 : >> # egrep ^mps0 /var/run/dmesg.boot >> mps0: >> mps0: Firmware: 20.00.04.00, Driver: 20.00.00.00-fbsd >>=20 >> FreeBSD 11-RC1 : >> # egrep ^mps0 /var/run/dmesg.boot >> mps0: >> mps0: Firmware: 20.00.04.00, Driver: 21.01.00.00-fbsd >>=20 >> Strangely enough there is no P21 driver/firmware @Avago : >> = http://www.avagotech.com/products/server-storage/host-bus-adapters/sas-921= 1-8i#downloads >> So I assume firmware P20 is the right one to use with FreeBSD 11. >> Well it is : = https://forums.freenas.org/index.php?threads/firmware-version-p21.45130/#p= ost-304399 > I have worked with those guys, and they consistently say there is no = difference in the firmware revs, but... I have seen major differences.. = The 21 sounds like an oem release. The .21 version is from FreeBSD. I just had a look, driver moved from .20 to .21 with this : https://svnweb.freebsd.org/base?view=3Drevision&revision=3D302031 This bugfix has also been committed to stable/10 : = https://github.com/freebsd/freebsd/commit/4542c2afc3d1697def1e77a37b7d0a77= 3ee40964 From owner-freebsd-fs@freebsd.org Mon Aug 22 20:23:05 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8C1B7BC299B for ; Mon, 22 Aug 2016 20:23:05 +0000 (UTC) (envelope-from the.lists@mgm51.com) Received: from oneyou.mcmli.com (oneyou.mcmli.com [IPv6:2607:f2f8:af30::100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "oneyou.mcmli.com", Issuer "RapidSSL SHA256 CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 721311688 for ; Mon, 22 Aug 2016 20:23:05 +0000 (UTC) (envelope-from the.lists@mgm51.com) Received: from sentry.24cl.com (sentry.24cl.com [IPv6:2001:558:6017:94:4cfb:c574:24a0:7961]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "sentry.24cl.com", Issuer "Mike's Certificate Authority" (verified OK)) by oneyou.mcmli.com (Postfix) with ESMTPS id 3sJ4kw2lQmz4D1q for ; Mon, 22 Aug 2016 16:23:04 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mgm51.com; s=mgm51-02; t=1471897384; bh=jutxa+OybglFOkkQeppzA3vQu8TBz9vwTD4KtBRc6CM=; h=Subject:To:From:Message-ID:Date; b=UoXzp7ZIU8/MSYE6kg0CHZRXJXQr+rwo5YTb/N0ck4hol6dynJ7Cram7qz6N5gHGS 3br5QR1wmU9EoCjJu1PhhZQRoZ5ZqT3OqJUrmziA1mP3OHL+Hq/LzYjvvDf1dgzav8 sN38UFMG1k5u7lwVVXkvzdSVCcpqWbaOB+45f7vLPtFBUtgD1PBO12u07in2dtWSOB Zust827M20LnUq9DaBsYRZV5vj/z3chICi9UDvQY/5IzXXZEF6zqKDDf8KPfGCwSPg t1EOValpTZi7e8KIlzDT3eCLJfavULcJPgcxSoMVG/aP2x4jfLIjqJ8t5py1YHCVYV URbheDH5wIn1A== Received: from [IPv6:fdcf:b715:2f4d:1:49a0:b0b7:349e:6605] (unknown [IPv6:fdcf:b715:2f4d:1:49a0:b0b7:349e:6605]) by sentry.24cl.com (Postfix) with ESMTP id 3sJ4kv1VzdzGHHD for ; Mon, 22 Aug 2016 16:23:03 -0400 (EDT) Subject: Re: Recommended HBA for ZFS, contemporary To: freebsd-fs@freebsd.org References: <306ccab3-4c78-751a-2258-54701a64ab4b@internetx.com> <41DE4DAA-B5E1-4BB9-AF54-780236A901EF@gmail.com> From: Mike Message-ID: <33322d71-1f14-38f5-a8d5-657c315e3a62@mgm51.com> Date: Mon, 22 Aug 2016 16:22:45 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 20:23:05 -0000 On 8/22/2016 2:45 PM, Karl Denninger wrote: > On 8/22/2016 13:38, Mike wrote: >> On 8/22/2016 2:36 PM, Karl Denninger wrote: >>> On 8/22/2016 13:34, Mike wrote: >>>> On 8/22/2016 12:18 PM, Ben RUBSON wrote: >>>>>> On 22 Aug 2016, at 18:14, InterNetX - Juergen Gotteswinter wrote: >>>>>> >>>>>> LSI SAS 2008 Based HBA, there are dozens of OEM available for Budget Prices >>>>> With the IT firmware. >>>> What's the significance of "IT firmware"? >>>> >>>> >>> It's (slightly) faster, provided you don't turn on any of the other >>> stuff (which is always dumb with these adapters.) >> Does it come with the HBA board(s), needs to be purchased or downloaded >> separately? > I've never gotten a card with it on it; I've always had to flash it. > Flashing these cards is a bit of a highwire act and can brick them, but > I've never had it happen myself. With that said my real reason to flash > all of mine is to have all of them on the same firmware just so I can > eliminate that as a possible contributor to problems if one starts > acting up and the others are not. It's more of a "good practice" thing > than a need IMHO; it's not a lot of fun to try to run down a problem > only to find out after much hair has been pulled out of one's own head > that the problem is bad firmware on a board! thx. From owner-freebsd-fs@freebsd.org Tue Aug 23 00:52:13 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54DB2BB7CD1 for ; Tue, 23 Aug 2016 00:52:13 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "wonkity.com", Issuer "wonkity.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 08FCC1CA4 for ; Tue, 23 Aug 2016 00:52:12 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.15.2/8.15.2) with ESMTPS id u7N0q7hK040431 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 22 Aug 2016 18:52:07 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.15.2/8.15.2/Submit) with ESMTP id u7N0q7dI040428; Mon, 22 Aug 2016 18:52:07 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Mon, 22 Aug 2016 18:52:07 -0600 (MDT) From: Warren Block To: Karl Denninger cc: freebsd-fs@freebsd.org Subject: Re: Recommended HBA for ZFS, contemporary In-Reply-To: <580a6ea8-9c84-3e99-5a7d-7b2434c84e67@denninger.net> Message-ID: References: <580a6ea8-9c84-3e99-5a7d-7b2434c84e67@denninger.net> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Mon, 22 Aug 2016 18:52:07 -0600 (MDT) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2016 00:52:13 -0000 On Mon, 22 Aug 2016, Karl Denninger wrote: > I have a number of the Intel 6-port expanders in production servers and > have never have had any trouble with them; they just plain work. The > only note I'll make on the expanders (and these host cards) is to make > sure you have sufficient airflow in the chassis; they're passive > heat-sink cooled so airflow matters. In any sort of proper server > chassis this is probably not going to be an issue but if you're going > for something that doesn't sound like an old 707 jet on take-off you > need to pay attention to make sure there's sufficient cooling airflow. There is an article on converting the Dell H200/H310 cards to LSI 9211 where they show a picture of a fan mounted on the heatsink but do not discuss it: https://techmattr.wordpress.com/2016/04/11/updated-sas-hba-crossflashing-or-flashing-to-it-mode-dell-perc-h200-and-h310/ I'm not happy with that article because it says "here is a CD of all the firmware and tools you need", but does not list sources to download them yourself . Call me crazy, but that makes me nervous. Anyway, I tried touching the heatsink of my new/used H200 while the card was idle, and yes, it gets surprisingly hot. Untouchably hot, some would say. A 40mm fan fits perfectly. The standard trick of using screws that just fit between the fins and bite in a little works. Like most, it was a 12V fan, but did not need that much power for this job. At 5V, it runs and keeps the heatsink cool enough to touch. I have another H200 to convert, and mean to document the process. I also have vendor sources for the files required. From owner-freebsd-fs@freebsd.org Tue Aug 23 05:11:16 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5F92ABC3491 for ; Tue, 23 Aug 2016 05:11:16 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5683D1389 for ; Tue, 23 Aug 2016 05:11:16 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u7N5BD3L027328 for ; Tue, 23 Aug 2016 05:11:16 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201608230511.u7N5BD3L027328@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-fs@FreeBSD.org Subject: Problem reports for freebsd-fs@FreeBSD.org that need special attention Date: Tue, 23 Aug 2016 05:11:16 +0000 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2016 05:11:16 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 203492 | mount_unionfs -o below causes panic Open | 136470 | [nfs] Cannot mount / in read-only, over NFS Open | 139651 | [nfs] mount(8): read-only remount of NFS volume d Open | 140068 | [smbfs] [patch] smbfs does not allow semicolon in Open | 144447 | [zfs] sharenfs fsunshare() & fsshare_main() non f Open | 203419 | solaris assert: (dn->dn_phys->dn_nlevels == 0 && 6 problems total for which you should take action. From owner-freebsd-fs@freebsd.org Tue Aug 23 07:29:27 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 09F64BC228A for ; Tue, 23 Aug 2016 07:29:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EE22A11C9 for ; Tue, 23 Aug 2016 07:29:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u7N7TQGA082842 for ; Tue, 23 Aug 2016 07:29:26 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 212043] panic when detaching USB mass storage Date: Tue, 23 Aug 2016 07:29:26 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: usb X-Bugzilla-Version: 11.0-RC1 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2016 07:29:27 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D212043 Andriy Gapon changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |trasz@FreeBSD.org Assignee|freebsd-usb@FreeBSD.org |freebsd-fs@FreeBSD.org --- Comment #3 from Andriy Gapon --- Bug #210316 seems to be related. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Tue Aug 23 07:47:00 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9DEEFBC2B71 for ; Tue, 23 Aug 2016 07:47:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 801441F47 for ; Tue, 23 Aug 2016 07:47:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u7N7l0wQ024178 for ; Tue, 23 Aug 2016 07:47:00 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 211381] L2ARC degraded, repeatedly, on Samsung SSD 950 Pro nvme Date: Tue, 23 Aug 2016 07:47:00 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: avg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2016 07:47:00 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D211381 Andriy Gapon changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |Closed Resolution|--- |FIXED --- Comment #14 from Andriy Gapon --- Assuming that the problem was fixed by https://svnweb.freebsd.org/base?view=3Drevision&revision=3D297848 --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Tue Aug 23 07:55:37 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B8FABC3130 for ; Tue, 23 Aug 2016 07:55:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3413F1BCA for ; Tue, 23 Aug 2016 07:55:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u7N7tYaS042429 for ; Tue, 23 Aug 2016 07:55:37 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 209158] node / npm triggering zfs rename deadlock Date: Tue, 23 Aug 2016 07:55:35 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: needs-qa, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: avg@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2016 07:55:37 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D209158 --- Comment #42 from commit-hook@freebsd.org --- A commit references this bug: Author: avg Date: Tue Aug 23 07:55:06 UTC 2016 New revision: 304671 URL: https://svnweb.freebsd.org/changeset/base/304671 Log: MFC r303763,303791,303869: zfs: honour and make use of vfs vnode locking protocol PR: 209158 Changes: _U stable/10/ stable/10/sys/cddl/compat/opensolaris/sys/vnode.h stable/10/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_dir.h stable/10/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_vfsops.h stable/10/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_znode.h stable/10/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_acl.c stable/10/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_dir.c stable/10/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_sa.c stable/10/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c stable/10/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c stable/10/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Tue Aug 23 07:58:06 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BC156BC32A2 for ; Tue, 23 Aug 2016 07:58:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AB7B91EA9 for ; Tue, 23 Aug 2016 07:58:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u7N7w6tM045659 for ; Tue, 23 Aug 2016 07:58:06 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 209158] node / npm triggering zfs rename deadlock Date: Tue, 23 Aug 2016 07:58:06 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: needs-qa, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: avg@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2016 07:58:06 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D209158 Andriy Gapon changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|In Progress |Closed --- Comment #43 from Andriy Gapon --- No new regressions have been reported in head and stable/11, so I've just merged this to stable/10. I am not planning MFC to stable/9 because of many merge conflicts. But if anyone does the merge and tests the result, then I= can commit it. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Tue Aug 23 08:44:24 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8EB0FBC179B; Tue, 23 Aug 2016 08:44:24 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id B19551FA4; Tue, 23 Aug 2016 08:44:23 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA01478; Tue, 23 Aug 2016 11:44:21 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1bc7Jk-0000nA-V8; Tue, 23 Aug 2016 11:44:20 +0300 To: FreeBSD Current , freebsd-fs From: Andriy Gapon Subject: major code change for .zfs Message-ID: <34c1e635-4f38-d483-bcf9-768311d7c726@FreeBSD.org> Date: Tue, 23 Aug 2016 11:43:00 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2016 08:44:24 -0000 Please review and test a change to .zfs code that is intended to make the code aligned with FreeBSD VFS and, as such, more stable: https://reviews.freebsd.org/D7421 The change removes two features. .zfs/shares is gone because it was unused on FreeBSD anyway. We can restore that when we need it. An ability to take a snapshot by creating a directory under .zfs/snapshot is removed. I hope that you didn't use it. Please do not start using it now :-) Again, this feature can be restored with some work. The reason I removed it is that its companion features of destroying and renaming snapshots were already missing on FreeBSD, and properly implementing the feature required some more work. -- Andriy Gapon From owner-freebsd-fs@freebsd.org Tue Aug 23 09:22:29 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B29E4BC24EE for ; Tue, 23 Aug 2016 09:22:29 +0000 (UTC) (envelope-from rincebrain@gmail.com) Received: from mail-ua0-x233.google.com (mail-ua0-x233.google.com [IPv6:2607:f8b0:400c:c08::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 699601779 for ; Tue, 23 Aug 2016 09:22:29 +0000 (UTC) (envelope-from rincebrain@gmail.com) Received: by mail-ua0-x233.google.com with SMTP id 97so234355587uav.3 for ; Tue, 23 Aug 2016 02:22:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=9WPlqeeidFFHDcHq7jJwap4sPd9CLRn1iQEmpCdCl3s=; b=B4IdB69vybcx3nZgltQOVID03KS6b705Hy4N3rcIRE2Aojvm/oeDU3KXynvOLMbd1/ Pc+O+/q+KqMWE/qK+HxfqscxdZsx+FqLE9a8PjqFKjd3vCQp5vre4rG0QcX8vLWZBSv1 1X/DzZB1xwLNAlz51kaspAmjiThhuhU2VA8gmApamIYTROSgM1uqkPSNGws4zzDWiCFX V3lESaTK6/YuLDcJqC1pIHc6MVmEsmnAjCnuyUAhXz2vGBvtJuhZX1Ll7LWhqRRvCogY RZQpGGTBZz92QtjYdnLZYDaI1bC82t6PXlCZ4X3vlplr5Ggp1ITOr+draku30lMJK1Qq GhTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=9WPlqeeidFFHDcHq7jJwap4sPd9CLRn1iQEmpCdCl3s=; b=ArVusT2fokk1njCpi/QqRtpS73oweoBkrujtEm7/qbZM1HrHxKc0F8EstlXyJ7DB0V n5fin618tBnJ7yhm0T7XmFTLMspS1lUpBRrETdv7nmXnbUMbGE1LtslAjpcvbLhghasK IUfvY+9d5TCvSla02MiUG5vYUX2LgxnYJC8AlpVBimfsFt5Mt2M+0knRam8enSuz+AMW v0ryUFRamBgyN87qI+VJ4W6MzQbEHOCKK8oqn/3sr9HRU9w9YUkvIMF/11d2vDExcH6b Mc8FqV4bF8nsGOfeJZIyw6+6hETUqJe0UCjmsN3IVCQNtPEWAIDXXudX8TK1MhaMjIBc jC3A== X-Gm-Message-State: AEkoouuvRD0QztIp4VLOf3BAE9HZ3s0D12HanvXqpGCX5Ph6EV0gJSZXW+zTi0JLoVPL71BXpcf+H0u59LQUdA== X-Received: by 10.176.68.7 with SMTP id m7mr14159247uam.44.1471944148640; Tue, 23 Aug 2016 02:22:28 -0700 (PDT) MIME-Version: 1.0 Received: by 10.103.82.204 with HTTP; Tue, 23 Aug 2016 02:22:27 -0700 (PDT) In-Reply-To: References: <580a6ea8-9c84-3e99-5a7d-7b2434c84e67@denninger.net> From: Rich Date: Tue, 23 Aug 2016 05:22:27 -0400 Message-ID: Subject: Re: Recommended HBA for ZFS, contemporary To: Warren Block Cc: Karl Denninger , freebsd-fs Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2016 09:22:29 -0000 The reason they don't provide a source is that Megarec is one of those tools that is theoretically only provided to OEMs or people who sign contracts with LSI (AFAIK), but other than that, the other tools are all readily available from LSI's site. (SAS2IRCU also used to be similarly hard to find, but they've fortunately stopped that.) I've just got my own FreeDOS image with sas2flsh, megarec for DOS, and megacli, and it works perfectly fine with those instructions. If it makes you itchy, cut your own FreeDOS image and lob those binaries from LSI's site and $OTHER_SOURCE for megarec and then go to town. (You don't need megarec for crossflashing between IR/IT on an HBA, though, that's solely for when you're convincing an "actual" RAID card that it's just a dumb HBA.) - Rich On Mon, Aug 22, 2016 at 8:52 PM, Warren Block wrote: > On Mon, 22 Aug 2016, Karl Denninger wrote: > > I have a number of the Intel 6-port expanders in production servers and >> have never have had any trouble with them; they just plain work. The >> only note I'll make on the expanders (and these host cards) is to make >> sure you have sufficient airflow in the chassis; they're passive >> heat-sink cooled so airflow matters. In any sort of proper server >> chassis this is probably not going to be an issue but if you're going >> for something that doesn't sound like an old 707 jet on take-off you >> need to pay attention to make sure there's sufficient cooling airflow. >> > > There is an article on converting the Dell H200/H310 cards to LSI 9211 > where they show a picture of a fan mounted on the heatsink but do not > discuss it: > > https://techmattr.wordpress.com/2016/04/11/updated-sas-hba- > crossflashing-or-flashing-to-it-mode-dell-perc-h200-and-h310/ > > I'm not happy with that article because it says "here is a CD of all the > firmware and tools you need", but does not list sources to download them > yourself . Call me crazy, but that makes me nervous. > > Anyway, I tried touching the heatsink of my new/used H200 while the card > was idle, and yes, it gets surprisingly hot. Untouchably hot, some would > say. A 40mm fan fits perfectly. The standard trick of using screws that > just fit between the fins and bite in a little works. Like most, it was a > 12V fan, but did not need that much power for this job. At 5V, it runs and > keeps the heatsink cool enough to touch. > > I have another H200 to convert, and mean to document the process. I also > have vendor sources for the files required. > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Tue Aug 23 09:34:16 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ABD39BC2C00 for ; Tue, 23 Aug 2016 09:34:16 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: from mail-wm0-x235.google.com (mail-wm0-x235.google.com [IPv6:2a00:1450:400c:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41B511F43 for ; Tue, 23 Aug 2016 09:34:16 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: by mail-wm0-x235.google.com with SMTP id i5so185553681wmg.0 for ; Tue, 23 Aug 2016 02:34:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Te4A8zd+KDwRMkGiBZX9GTSQIKuO/pR8y8PcJIdpSuU=; b=A+Ztb64kGUq5WZYJsI09r/xyRMeecJtry/Yujg+chwE8AHUoFuVIa8dgJgwS9pMCDX OO80u+334A/KvyrmpdBRTF60eMwZpJ3Le9GuSsjTfxF7qkrp9yrGZ04uETtlc5oWyr8+ agGsd5jSnWClqLEsatNdImzGhQLQ/vhOgqmwFPJ7vkFtsLjZmSL+VaOR6ySneYSLB2sg UEKSFaxhyJf4ERDDgyrhOfHWUcYsOzI+psA+MVX7ycbmBW80O+/kzFVusSEcZ+5sKZuG 6cxtvV1lWHUAvdWCrjxOKApeG4iYkfzpRxJ/2rxD+aXtPyS0+V7urkGF3iEHjcoW9kIS S/2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Te4A8zd+KDwRMkGiBZX9GTSQIKuO/pR8y8PcJIdpSuU=; b=lJ9cBTc6sHkUj1CwE8wHIenr3GbOhc6/O/4hao6aW/MVbEeCFkEiMsOS17ozZ7p9D3 2DMzN665BvbxgNMAstTiZL+0DXAbxblg1/rr6SBZskJ/80RChSPmdmKTXJM9e6BNKPW+ BOftrCBCwaHBVMXy8qFljuYdeVH7wyg7wGHMd4KaNx9Ecmt5Usv6VRQMwHplE4agfhev wPE9tvytmJQSDhs4/eQYyqiB4iF/MxOLW/AZVYykdFiWJHlZ2A+L8A8x5urhpUWxA+pl LnojcHm7C9BCF8BW8KS0ePDOD/pcfO+O2Ydv84ietJgqUWO0fcrmr5SyRqUokm3Pb3QH Fh5w== X-Gm-Message-State: AEkooutKdIgf4DuMd76qWWMO5PCm5mvqQUD1rMYhQVjABDtH4ZR1zbsV+eaJX9wN8o+0BuynMXImV+ngITUj1Q== X-Received: by 10.28.220.8 with SMTP id t8mr20517836wmg.114.1471944854848; Tue, 23 Aug 2016 02:34:14 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.54.202 with HTTP; Tue, 23 Aug 2016 02:34:14 -0700 (PDT) In-Reply-To: <938E209E-143E-4399-809F-006E455E1C9D@kraus-haus.org> References: <57B59CBC.8000904@norma.perm.ru> <938E209E-143E-4399-809F-006E455E1C9D@kraus-haus.org> From: krad Date: Tue, 23 Aug 2016 10:34:14 +0100 Message-ID: Subject: Re: zpool list FREE vs zfs list AVAIL To: Paul Kraus Cc: "Eugene M. Zheganin" , FreeBSD FS Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2016 09:34:16 -0000 zpool shows raw storage, ie doesnt take into account the raid levels redundancy requirements. zfs shows what a file system/dateset can use. Things like quotas can make this different than the pool size. I think it also takes into consideration the raid requirements, but i may be wrong there. df, can start to go a bit mad when you enable compression and dedup, as the projected capacities can actually increase depending on the compression and dedup ratios you are achieving. In the case you state its probably a quota as the mismatch is so big. On 21 August 2016 at 01:27, Paul Kraus wrote: > > > On Aug 18, 2016, at 7:32 AM, Eugene M. Zheganin > wrote: > > > > What is the difference between zpool list FREE for a pool and zfs list > > AVAIL ? Because they differ a lot, I'm looking at a server at the moment > > where the difference is like dozens of times: zfs list reports that 97 > > gigabytes is available, and the zpool list for the same pool says that > > 4.18 terabytes is free. From my point of view this should be the same > > number. > > Please provide complete zpool status, zpool list, and zfs list output. > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Tue Aug 23 11:09:15 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B44EBC30FF for ; Tue, 23 Aug 2016 11:09:15 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id 5293D1820 for ; Tue, 23 Aug 2016 11:09:13 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Tue, 23 Aug 2016 13:09:07 +0200 Authentication-Results: connect.ultra-secure.de; auth=pass (login); spf=none smtp.mailfrom=ultra-secure.de Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 127.0.0.10 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=127.0.0.10; helo=connect.ultra-secure.de; envelope-from= Received: from connect.ultra-secure.de (webmail [127.0.0.10]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id 1B7D392A-1737-4A48-9B42-790E11C2D9EB.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=AES256-SHA verify=NO); Tue, 23 Aug 2016 13:09:01 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Tue, 23 Aug 2016 13:09:00 +0200 From: rainer@ultra-secure.de To: Rich Cc: Warren Block , freebsd-fs Subject: Re: Recommended HBA for ZFS, contemporary In-Reply-To: References: <580a6ea8-9c84-3e99-5a7d-7b2434c84e67@denninger.net> Message-ID: <7914465331af14cde7fe43d4788cd336@ultra-secure.de> X-Sender: rainer@ultra-secure.de User-Agent: Roundcube Webmail/1.2.0 X-Haraka-GeoIP: --, , NaNkm X-Haraka-GeoIP-Received: X-Haraka-p0f: os="undefined undefined" link_type="undefined" distance=undefined total_conn=undefined shared_ip=Y X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 34, bad: 0, connections: 34, history: 34, pass:all_good, relaying X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2016 11:09:15 -0000 Am 2016-08-23 11:22, schrieb Rich: > The reason they don't provide a source is that Megarec is one of those > tools that is theoretically only provided to OEMs or people who sign > contracts with LSI (AFAIK), but other than that, the other tools are > all > readily available from LSI's site. (SAS2IRCU also used to be similarly > hard > to find, but they've fortunately stopped that.) > > I've just got my own FreeDOS image with sas2flsh, megarec for DOS, and > megacli, and it works perfectly fine with those instructions. If it > makes > you itchy, cut your own FreeDOS image and lob those binaries from LSI's > site and $OTHER_SOURCE for megarec and then go to town. > > (You don't need megarec for crossflashing between IR/IT on an HBA, > though, > that's solely for when you're convincing an "actual" RAID card that > it's > just a dumb HBA.) It's relatively important to know that only the DOS-version, and only P14 (and below, probably) allows cross-flashing HBAs). The tutorials don't really say that, they kind-of assume that you will use the DOS-binaries anyway. In my IT-career, I've really only touched DOS a very small amount of time. Imagine my aversion at creating a DOS boot-disk ;-) From owner-freebsd-fs@freebsd.org Tue Aug 23 13:01:58 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 27CA9BC3CB2 for ; Tue, 23 Aug 2016 13:01:58 +0000 (UTC) (envelope-from lists@peter.de.com) Received: from elsa.gfuzz.de (elsa.gfuzz.de [176.9.193.218]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E26CD16C2 for ; Tue, 23 Aug 2016 13:01:57 +0000 (UTC) (envelope-from lists@peter.de.com) Received: from localhost (localhost [127.0.0.1]) by elsa.gfuzz.de (Postfix) with ESMTP id DD21F1000CD for ; Tue, 23 Aug 2016 14:52:10 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at elsa.gfuzz.de Received: from elsa.gfuzz.de ([127.0.0.1]) by localhost (localhost [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6UOx9W9ZJIdQ for ; Tue, 23 Aug 2016 14:52:09 +0200 (CEST) Received: from mail.opdns.de (ipbcc23b2b.dynamic.kabel-deutschland.de [188.194.59.43]) (Authenticated sender: oliver@gfuzz.de) by elsa.gfuzz.de (Postfix) with ESMTPSA id 74E9E10006E for ; Tue, 23 Aug 2016 14:52:09 +0200 (CEST) Date: Tue, 23 Aug 2016 14:52:07 +0200 From: Oliver Peter To: FreeBSD FS Subject: Re: [ZFS] "online" spares ? Message-ID: <20160823125207.GA20287@mail.opdns.de> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: Linux 3.16.0-4-686-pae i686 User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2016 13:01:58 -0000 On Mon, Aug 22, 2016 at 06:31:29PM +0200, Ben RUBSON wrote: > Hello, > > Let's assume the following status : > > pool: zroot > state: ONLINE > config: > NAME STATE READ WRITE CKSUM > zroot ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > (...) > spares > 15751675431801904320 REMOVED was /dev/label/X92020TYEEGHT > > For some reason one of the spares is in removed state but is now available again to the system. > "online" command does not work for spares. > Is there any way to bring spares back to available state without having to "remove" and "add" them again ? > > Many thanks ! If the spare comes up as /dev/label/X92020TYEEGHT again you might want to give 'zpool clear zroot' a try. That did it for me. -- Oliver PETER oliver@gfuzz.de 0x456D688F From owner-freebsd-fs@freebsd.org Tue Aug 23 13:03:52 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EAD72BC3DF3 for ; Tue, 23 Aug 2016 13:03:52 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x241.google.com (mail-wm0-x241.google.com [IPv6:2a00:1450:400c:c09::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F24C17CF for ; Tue, 23 Aug 2016 13:03:52 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x241.google.com with SMTP id q128so18067137wma.1 for ; Tue, 23 Aug 2016 06:03:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=SXcXXoV9DQZpv5CttmNNRLCSdJiwICficky0oWGAquc=; b=PLCQ66+gwe8Q3p87kIsmk0sR85+FbUnYdudVS/jPfRUSCSvBZrYI5hEaKRSwlagnkN FQyDZh7Qw8zP62lzj9EjYMjNsEzcY8p3s+cdnqFK0wSLjyxnpEDtx6LFqJUh59KkgEzq wM22FJFnbGQYP5XW7/g/ss51lyUKsHyTPNQd5ivb09K4RLopiJvRoHgTETGccIXhl3BV wvvvGOX0C148P45EgTyYvitvYnIs9SdOG+CZ6hjS2WAJlwSPMwiQAbKFV0/ZqRNgvE26 yYGUQXX6t8o9DX3qc0HbxFQB6J8ciUZ7LOu4WItNcYXA38e8Vkhocqze1TH5d1Qo/SE0 DAxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=SXcXXoV9DQZpv5CttmNNRLCSdJiwICficky0oWGAquc=; b=gJxfEyDHEZRgM26f4Nl0Vdlo0ZlUxjkQFW+89VxVutvbrPJM1X9DEU5WOzOTgGCl2N rHksn0SV6irABDgh9bLsyE0OgnBImF9c90L2fF89iLMuXaBTljdYIJbHJCVzNdm656OY RwGo1FIwytIpVEYcUny6mWyNDp2BOeuyJI2lXUWLmgN6JYky2C8a/K7fi/A0zLUZEu0V B5vf/jdn+uz34IV6sj28tX048CRRQq3EJj7CK1Pr+SVFtG5738bSvHge9PN5fXUcJ4aw UVJHLWBuDwKJpjRriDEhXuKqbehEY5PpwYj6pjZwcImxCnmS5ZXzgaALiFfLmyz0NiN6 Q0aw== X-Gm-Message-State: AEkooutSjvIsaWwoLcVUiNZUBy3jTqyO5qPS9c/mtun/qXVz1Ux8hortSd6wvP37Y6EdKQ== X-Received: by 10.28.4.77 with SMTP id 74mr21470554wme.84.1471957430394; Tue, 23 Aug 2016 06:03:50 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (ANice-651-1-185-161.w83-197.abo.wanadoo.fr. [83.197.121.161]) by smtp.gmail.com with ESMTPSA id ko7sm3671443wjc.48.2016.08.23.06.03.49 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 23 Aug 2016 06:03:49 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: [ZFS] "online" spares ? From: Ben RUBSON In-Reply-To: <20160823125207.GA20287@mail.opdns.de> Date: Tue, 23 Aug 2016 15:03:48 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <0BE04DA7-1C96-436D-9E99-6E25E339E364@gmail.com> References: <20160823125207.GA20287@mail.opdns.de> To: FreeBSD FS X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2016 13:03:53 -0000 > On 23 Aug 2016, at 14:52, Oliver Peter wrote: >=20 > On Mon, Aug 22, 2016 at 06:31:29PM +0200, Ben RUBSON wrote: >> Hello, >>=20 >> Let's assume the following status : >>=20 >> pool: zroot >> state: ONLINE >> config: >> NAME STATE READ WRITE CKSUM >> zroot ONLINE 0 0 0 >> mirror-0 ONLINE 0 0 0 >> (...) >> spares >> 15751675431801904320 REMOVED was = /dev/label/X92020TYEEGHT >>=20 >> For some reason one of the spares is in removed state but is now = available again to the system. >> "online" command does not work for spares. >> Is there any way to bring spares back to available state without = having to "remove" and "add" them again ? >>=20 >> Many thanks ! >=20 > If the spare comes up as /dev/label/X92020TYEEGHT again you might want = to > give 'zpool clear zroot' a try. That did it for me. Oliver, many thanks, it worked ! Thank you again, Ben From owner-freebsd-fs@freebsd.org Tue Aug 23 15:40:13 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 43D63BC38A9 for ; Tue, 23 Aug 2016 15:40:13 +0000 (UTC) (envelope-from the.lists@mgm51.com) Received: from oneyou.mcmli.com (oneyou.mcmli.com [IPv6:2607:f2f8:af30::100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "oneyou.mcmli.com", Issuer "RapidSSL SHA256 CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 296BB1AAA for ; Tue, 23 Aug 2016 15:40:13 +0000 (UTC) (envelope-from the.lists@mgm51.com) Received: from sentry.24cl.com (sentry.24cl.com [IPv6:2001:558:6017:94:4cfb:c574:24a0:7961]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "sentry.24cl.com", Issuer "Mike's Certificate Authority" (verified OK)) by oneyou.mcmli.com (Postfix) with ESMTPS id 3sJZQ41GQpz4D2q for ; Tue, 23 Aug 2016 11:40:12 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mgm51.com; s=mgm51-02; t=1471966812; bh=AdM9cG0vtyxrz/a14EipkUOQuTNQNQSrTyKHT/DMByk=; h=Subject:To:From:Message-ID:Date; b=sqmuyq63E85ql2DNp6ZtzhON4XLeSiymWC69dc8cJ/2BxOR5QcED5r1+UUn0P+6Vl qVMhiYFVqfzBaDneLqYVGLdjrTyA/pZGTcapL56BZlaKjfx4DYdDzBOgsIZ+ReAHL7 nqHFJqikKGAqPrx4tsi+w3vdkZ5+25ldnpM1yl8GT+nAOJaCB6a0XNE1cawRFpdyuX ATatvtelhOMlDo2xrbgqy0+5E17C+Uzc0dWpr8t6Gs64LHk5SdL2EEk2XF1VX2t5SY C+m8/4MY+SUP9ldJX4rw9lFRjpRyDhuw1/7nqaoUS+pq71TGUqqAhw5Wopcxqzm3Y3 VJViCYj+lTOZg== Received: from [IPv6:fdcf:b715:2f4d:1:4478:a60a:b242:5f6e] (unknown [IPv6:fdcf:b715:2f4d:1:4478:a60a:b242:5f6e]) by sentry.24cl.com (Postfix) with ESMTP id 3sJZQ26qwkzGHHD for ; Tue, 23 Aug 2016 11:40:10 -0400 (EDT) Subject: Re: Recommended HBA for ZFS, contemporary To: freebsd-fs@freebsd.org References: <580a6ea8-9c84-3e99-5a7d-7b2434c84e67@denninger.net> <7914465331af14cde7fe43d4788cd336@ultra-secure.de> From: Mike Message-ID: <67cebd8d-e33d-56a9-7458-1cee700a7e15@mgm51.com> Date: Tue, 23 Aug 2016 11:39:51 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <7914465331af14cde7fe43d4788cd336@ultra-secure.de> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2016 15:40:13 -0000 On 8/23/2016 7:09 AM, rainer@ultra-secure.de wrote: > [snip] > > It's relatively important to know that only the DOS-version, and only > P14 (and below, probably) allows cross-flashing HBAs). > > The tutorials don't really say that, they kind-of assume that you will > use the DOS-binaries anyway. > > In my IT-career, I've really only touched DOS a very small amount of > time. Imagine my aversion at creating a DOS boot-disk ;-) DOS ain't so bad. I still have the 1991 full-backup for my DOS work PC. OS, word processor, c compiler, editor, plus all the apps' source I was developing. Backup file is 7MB. :) From owner-freebsd-fs@freebsd.org Tue Aug 23 21:19:40 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 796AABC4BCE for ; Tue, 23 Aug 2016 21:19:40 +0000 (UTC) (envelope-from lkateley@kateley.com) Received: from mail-it0-x22b.google.com (mail-it0-x22b.google.com [IPv6:2607:f8b0:4001:c0b::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 463F71D13 for ; Tue, 23 Aug 2016 21:19:40 +0000 (UTC) (envelope-from lkateley@kateley.com) Received: by mail-it0-x22b.google.com with SMTP id e63so173296350ith.1 for ; Tue, 23 Aug 2016 14:19:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kateley-com.20150623.gappssmtp.com; s=20150623; h=reply-to:subject:references:to:from:organization:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=jXPjSBQ6+fQRgLKnAWHbkEN7JN+vbooqd3EqRKBboLA=; b=r2VxieGdqRi4ohLhyIqTQGyV6Jz3SbGqY9S/fqpMFIbJ97b7NYIFqmhLJ8zI+yoC8j cV/a85IWJAq4XIiyHvoO6YWaJ3IC3HOWyLCFrKf2gokXCzzJyNpDWD7CPnyisfqIQXiy Qt08/CnRwMIonkSUTH22Xgc0RxBcbJOlOH4kXG8m8Ax65gFHuHaobJBBqxiH9lHATX6h 4OCQxd114xCUomweTgHoUyNTuxgVC7dY5drrywEepPqnOHEuzLTfBetJ3pBzzsw3QTp0 YQkTUumyMC8pf5SxUWYN/Xr49XAdt8Y6J30CTyWE6RmM/iIdbzdrNc08Aor1nL5Uu+Mf eKOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:reply-to:subject:references:to:from:organization :message-id:date:user-agent:mime-version:in-reply-to :content-transfer-encoding; bh=jXPjSBQ6+fQRgLKnAWHbkEN7JN+vbooqd3EqRKBboLA=; b=lHIospFcnptpK1aZT3fwgN2/e5xT+c2OYreASBh7x5I74YeR+xqcGFPDWtLQB7zKfT bOZdCRyclI8ABgcyd7mnIw0tuvt5ssMBkHAnj00W4aXczlRf51kMUxZgXK2NZD0z6lum 9UYV40zuwFDi+JWR8f2txiN6nSyjIw1a3rKIhLCSGOnMAidCmDwSPRVcZXHRx7bbI4ec iD2xdcBQbYB1f+KlE3E2xJF0A0C3tzHpXhozOmO6OASB3dNH8bc47/raFje0QS469AM/ WoLBgtjEz8agJsgSDR54X/cmuiCRwuAEqOZ2hddke/PE4nKrC2UUXzNqzSbNjzkEb7fw yenQ== X-Gm-Message-State: AEkoouu/r/8qyZNtzGR95rOu2wiB7M+KPRrJC7fw14Spxhwha5hT0KNOn4YRr52HxAUt3A== X-Received: by 10.107.14.74 with SMTP id 71mr46628ioo.93.1471987179276; Tue, 23 Aug 2016 14:19:39 -0700 (PDT) Received: from [192.168.0.19] (67-4-134-32.mpls.qwest.net. [67.4.134.32]) by smtp.googlemail.com with ESMTPSA id l19sm2095383iod.10.2016.08.23.14.19.38 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 23 Aug 2016 14:19:38 -0700 (PDT) Reply-To: linda@kateley.com Subject: Re: Recommended HBA for ZFS, contemporary References: <580a6ea8-9c84-3e99-5a7d-7b2434c84e67@denninger.net> <7914465331af14cde7fe43d4788cd336@ultra-secure.de> To: freebsd-fs@freebsd.org From: Linda Kateley Organization: Kateley Company Message-ID: Date: Tue, 23 Aug 2016 16:19:37 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <7914465331af14cde7fe43d4788cd336@ultra-secure.de> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2016 21:19:40 -0000 I have almost always used these instructions with 100% success http://brycv.com/blog/2012/flashing-it-firmware-to-lsi-sas9211-8i/ On 8/23/16 6:09 AM, rainer@ultra-secure.de wrote: > Am 2016-08-23 11:22, schrieb Rich: >> The reason they don't provide a source is that Megarec is one of those >> tools that is theoretically only provided to OEMs or people who sign >> contracts with LSI (AFAIK), but other than that, the other tools are all >> readily available from LSI's site. (SAS2IRCU also used to be >> similarly hard >> to find, but they've fortunately stopped that.) >> >> I've just got my own FreeDOS image with sas2flsh, megarec for DOS, and >> megacli, and it works perfectly fine with those instructions. If it >> makes >> you itchy, cut your own FreeDOS image and lob those binaries from LSI's >> site and $OTHER_SOURCE for megarec and then go to town. >> >> (You don't need megarec for crossflashing between IR/IT on an HBA, >> though, >> that's solely for when you're convincing an "actual" RAID card that it's >> just a dumb HBA.) > > > It's relatively important to know that only the DOS-version, and only > P14 (and below, probably) allows cross-flashing HBAs). > > The tutorials don't really say that, they kind-of assume that you will > use the DOS-binaries anyway. > > In my IT-career, I've really only touched DOS a very small amount of > time. Imagine my aversion at creating a DOS boot-disk ;-) > > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Wed Aug 24 20:43:17 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 51F2BBC58DE for ; Wed, 24 Aug 2016 20:43:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 40D181E13 for ; Wed, 24 Aug 2016 20:43:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u7OKhHUe066195 for ; Wed, 24 Aug 2016 20:43:17 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 203588] [ufs] fsck_ufs segfaults during journals after powerloss Date: Wed, 24 Aug 2016 20:43:17 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Not Enough Information X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution cc bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2016 20:43:17 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D203588 Kirk McKusick changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |Not Enough Information CC| |mckusick@FreeBSD.org Status|New |Closed --- Comment #2 from Kirk McKusick --- As noted this report does not have enough information to reproduce the prob= lem. Several bug fixes have been made to fsck that may have been relevant to this report. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Aug 25 09:09:12 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5BCEABC585E for ; Thu, 25 Aug 2016 09:09:12 +0000 (UTC) (envelope-from knswanso@andrew.cmu.edu) Received: from multidiff.com (unknown [IPv6:2001:41d0:2:dea3::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 277331438 for ; Thu, 25 Aug 2016 09:09:12 +0000 (UTC) (envelope-from knswanso@andrew.cmu.edu) Received: from localhost (localhost.localdomain [127.0.0.1]) by multidiff.com (Postfix) with ESMTP id D96B51A1BA0; Thu, 25 Aug 2016 06:50:07 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at multidiff.com Received: from multidiff.com ([127.0.0.1]) by localhost (ns3351455.ip-46-105-111.eu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Wf8Av1T8LJ5D; Thu, 25 Aug 2016 06:50:05 +0200 (CEST) Received: from pesp.com (unknown [77.70.18.227]) (Authenticated sender: contact@b2vauto.fr) by multidiff.com (Postfix) with ESMTPSA id F04936344AA8; Thu, 25 Aug 2016 05:52:17 +0200 (CEST) From: knswanso To: "freebsd-fs" , "gcc-help-owner" , "fsoc" , "gcc" Subject: Re: looks nice Date: Thu, 25 Aug 2016 06:52:16 +0300 Message-ID: <0000d53a5a9a$f13cc2d8$3d3d368d$@andrew.cmu.edu> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook, Build 17.551210 Thread-Index: AdHsNpubIgaqytrGEGyOt2aJEwU9fA== Content-Language: de Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2016 09:09:12 -0000 Greetings, I've recently came accross that amazing stuff, it looks nice I think, ytake a look Warmest, knswanso From owner-freebsd-fs@freebsd.org Fri Aug 26 05:19:37 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DF4C8BC59F6 for ; Fri, 26 Aug 2016 05:19:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CD9041BFD for ; Fri, 26 Aug 2016 05:19:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u7Q5Jbbw041894 for ; Fri, 26 Aug 2016 05:19:37 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 212168] [panic] [UFS] use-after-free panic (0xdeadc0dedeadc0de) Date: Fri, 26 Aug 2016 05:19:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-RC1 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: gjb@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2016 05:19:38 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D212168 Glen Barber changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Aug 26 09:39:06 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7DB4CA94DF6 for ; Fri, 26 Aug 2016 09:39:06 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x243.google.com (mail-wm0-x243.google.com [IPv6:2a00:1450:400c:c09::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 13DB3C75 for ; Fri, 26 Aug 2016 09:39:06 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x243.google.com with SMTP id o80so10627123wme.0 for ; Fri, 26 Aug 2016 02:39:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-transfer-encoding:subject:message-id:date:to :mime-version; bh=IynC9BxDzgbhQCuyCdPZfPZ0+RHs6P0WsO20O5/CFmE=; b=bvNLiJ99dSSphUChg4jes5uwdnH5RyTi9pKrrWxZtztBYsubbTlFv5Q1SKp3eH9VAJ SjkpbzQUyb0nxmmUlteZ6WnipitoIW1bdDlykqmpW9b7QilDWdgavazS9Nrph4rlWXbs W7uJvp4acBWyBkDtbiyNKkoOhy0t8oHPLsEQCC3iT77CXjs5kBDkHLiC4Os6jFI3rlfd 4eo5RBp9sk/TY7IiV68Wxn54YggB0oN/LAJiC1z9a5NU0WRy5nJ5VYPu9hBGYeQWiK+Q WY0X4VhvA2WBJ//qVrmMv8jq5ljLbd29xmQ/b9NebL+CGPI3gSqmmllPgGzmeYKPIOKR Zwew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:content-transfer-encoding:subject :message-id:date:to:mime-version; bh=IynC9BxDzgbhQCuyCdPZfPZ0+RHs6P0WsO20O5/CFmE=; b=Rx3kiimoE1LU/n5evuEEeEef2BoQiraKQ2zM9b6dyI6juDADKxNanX3pa5AD+6yjpg AAYfLlhmxdf+2SDVk1QQwIyKkexa3TJx9Eul3GacHslklIU+32/raxdUnUecLWtbg4rE SHsomwYLqLHZTHVkkU9bgrOSkB38VfbkDZ7mzJMhPvHqmUVtXSwtCOSjswluofk5BbNh RbEI6RL5yPqS8VmnkOq56hmNg6/Yqt9JxJ1anFbcYRB2ChKGq5VWc1GDug6uQebFf01p LFFtkR6O3r7MkT/Eqn3vVSBk3X1PjgTMhMsS+DN+rMZNu9oNnnIljakoS4G1ijZddCJp /mdA== X-Gm-Message-State: AE9vXwOMtkGZf/TUA1wgPDS9SnChU2bUggXCl6TpOrns9GFaUzjJBym2QDyxI5jV/Jx+zg== X-Received: by 10.28.223.9 with SMTP id w9mr2874470wmg.65.1472204344290; Fri, 26 Aug 2016 02:39:04 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (ANice-651-1-185-161.w83-197.abo.wanadoo.fr. [83.197.121.161]) by smtp.gmail.com with ESMTPSA id a2sm19120685wjg.46.2016.08.26.02.39.03 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 26 Aug 2016 02:39:03 -0700 (PDT) From: Ben RUBSON Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: [ZFS] ARC accounting bug ? Message-Id: Date: Fri, 26 Aug 2016 11:39:02 +0200 To: FreeBSD FS Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2016 09:39:06 -0000 Hello, Before opening a bug report, I would like to know whether what I see is = "normal" or not, and why. ### Test : # zfs import mypool # zfs set primarycache=3Dmetadata mypool # while [ 1 ]; do find /mypool/ >/dev/null; done real 0m59.469s user 0m0.243s sys 0m3.184s real 0m1.445s user 0m0.181s sys 0m1.264s real 0m1.441s user 0m0.150s sys 0m1.291s (...) // then all results are equal to 0m1.4s # zfs-mon -a ZFS real-time cache activity monitor Seconds elapsed: 162 Cache hits and misses: 1s 10s 60s tot ARC hits: 79228 76030 73865 74953 ARC misses: 22510 22184 21647 21955 ARC demand data hits: 0 0 0 0 ARC demand data misses: 4 7 8 7 ARC demand metadata hits: 79230 76030 73865 74953 ARC demand metadata misses: 22506 22177 21639 21948 ZFETCH hits: 47 29 32 31 ZFETCH misses:101669 98138 95433 96830 Cache efficiency percentage: 10s 60s tot ARC: 77.41 77.34 77.34 ARC demand data: 0.00 0.00 0.00 ARC demand metadata: 77.42 77.34 77.35 ZFETCH: 0.03 0.03 0.03 ### Question : I don't understand why I have so many ARC misses. There is no other activity on the server (as soon as I stop the find = loop, no more ARC hits). As soon as the first find loop is done, there is no more disk activity = (according to zpool instate -v 1), no read/write operations on mypool. So I'm pretty sure all metadata comes from ARC. So why are there so many ARC misses ? Any clue ? An accounting bug ? Many thanks ! Ben From owner-freebsd-fs@freebsd.org Fri Aug 26 09:40:13 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1EA82A94E47 for ; Fri, 26 Aug 2016 09:40:13 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x22f.google.com (mail-wm0-x22f.google.com [IPv6:2a00:1450:400c:c09::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A7B9BD1C for ; Fri, 26 Aug 2016 09:40:12 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x22f.google.com with SMTP id o80so110442463wme.1 for ; Fri, 26 Aug 2016 02:40:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=U/SgZpAjWHfWlReJQlJTvCV2OfiUblyMfwIhSnpPOu0=; b=avyQjussskl5yv+lZdlu1t8g0AN0ZozPC10ag0lawn1p50wP2PZrpQz2HxkbM8OtMd +VgtEjl6WtFNxBRrlIeG9rWdi5+UfShGvMDv943rjj4oFEQAzeve4xI4uXkdkS4iELIx TSVsdBU+VHAsKWIFQcx53M53jjVTCwdwcSc9GCf0voDfzY7EfXF7LD45Kf5OwEJ9T6O6 JaxPLoF33IedOZz4n+F2+kzx9xhoHN5Hpa6uJu5RxNuG4TKk0aebraCCSq+jyDu+cjWD JYlyMXJSZhHspffWCtR3+0JMqSIye1ssn+R3igMJwlbs44YftkEM6riNO9LWfXE/iFFs T+vA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=U/SgZpAjWHfWlReJQlJTvCV2OfiUblyMfwIhSnpPOu0=; b=QcuH7viPuV4rxVJeNK/luq3sZAfMjI4zuCiutE8k3LIIy93H0B/oYhz5L23e1r2a6a swU7aYO4GK07wKDLtqY2t8gn2VQukM6deYAp879BmXNtT8kJ4YjUxvbLgTxNlpOIhmYO 4PQB/VEHUcG5nHlstGX3l7gpRwsUReGSHg4RH6kqH+EZP3wJw3WU9srx8dEbLAw2Z6Tk sWqhetghjNpWBGuNHasj1sbZ5IyWDKX4JS+aUS52H99DQTwnXu/fc8g52NqElrWbXZXe uJmU3JMthYTvfgz+nUFWMn/BYnBgrcznEtU3SZ5r/T5f4qhK2TLsvCXh8J8XOuXNhpgI jUwA== X-Gm-Message-State: AE9vXwOyKH7yWKdKqn4zksz9lVL6maf3SOVKNgskOYn39BHyqPvAbR/yAL3dragAFLzaaw== X-Received: by 10.28.97.2 with SMTP id v2mr2682997wmb.31.1472204411011; Fri, 26 Aug 2016 02:40:11 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (ANice-651-1-185-161.w83-197.abo.wanadoo.fr. [83.197.121.161]) by smtp.gmail.com with ESMTPSA id qe2sm19161168wjc.28.2016.08.26.02.40.10 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 26 Aug 2016 02:40:10 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: [ZFS] ARC accounting bug ? From: Ben RUBSON In-Reply-To: Date: Fri, 26 Aug 2016 11:40:09 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <5B8C6663-5D21-4C03-AC69-CB7E70D3A6C2@gmail.com> References: To: FreeBSD FS X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2016 09:40:13 -0000 uname -v FreeBSD 11.0-RC2 #0 r304729: Wed Aug 24 06:59:03 UTC 2016 = root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC From owner-freebsd-fs@freebsd.org Fri Aug 26 12:01:41 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1AF35A938D7 for ; Fri, 26 Aug 2016 12:01:41 +0000 (UTC) (envelope-from juergen.gotteswinter@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D3506B50 for ; Fri, 26 Aug 2016 12:01:40 +0000 (UTC) (envelope-from juergen.gotteswinter@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id 44CC745FC0FD; Fri, 26 Aug 2016 14:01:32 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id k-86W1C+YTlm; Fri, 26 Aug 2016 14:01:30 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id 30F0E45FC0F0; Fri, 26 Aug 2016 14:01:30 +0200 (CEST) Reply-To: juergen.gotteswinter@internetx.com Subject: Re: [ZFS] ARC accounting bug ? References: <5B8C6663-5D21-4C03-AC69-CB7E70D3A6C2@gmail.com> To: Ben RUBSON , FreeBSD FS From: InterNetX - Juergen Gotteswinter Organization: InterNetX GmbH Message-ID: <067ac0ed-1f2d-c9fb-de4d-ed5da62fe10c@internetx.com> Date: Fri, 26 Aug 2016 14:01:28 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <5B8C6663-5D21-4C03-AC69-CB7E70D3A6C2@gmail.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2016 12:01:41 -0000 no bug, everything like i whould expect Am 26.08.2016 um 11:40 schrieb Ben RUBSON: > uname -v > FreeBSD 11.0-RC2 #0 r304729: Wed Aug 24 06:59:03 UTC 2016 root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Fri Aug 26 12:06:34 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 73EABA93BD5 for ; Fri, 26 Aug 2016 12:06:34 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x235.google.com (mail-wm0-x235.google.com [IPv6:2a00:1450:400c:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 07619E02 for ; Fri, 26 Aug 2016 12:06:34 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x235.google.com with SMTP id q128so276592947wma.1 for ; Fri, 26 Aug 2016 05:06:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=o8Dt61YLsH4S9hFNBao/bYKLJAxOvyBy3+9UBrERRWI=; b=FDTNz8s3wyCb+fFZdIcMWVVp+SsTg00WLCEWPXVaBujt0A5CDCNOOTtLtrs7PcLm2E f1gqcXlD6GO8gqSa4R0VDAMy+BKjg6xep/2IXHsWX3koRG4gID3xTlsnhbG86+ELOUwj VuiFZW5iRwfcWhcQPbFNVH85gPA2CaftSfVSEu++iQ6zpkt4yEsTvCIsBz5xgWUUCn7C AXi6Zf4DNu0L42qJtTB/mHi8H1Vk+5dZdXO4JfsiYhhmZDUQBFsM+aB4xbfmn+3MyMer nzq9jLH0zzoBgkMstN4QesKmv9X5zD0G2j0n18x0AsWnsdYTXtxifzfdxNCypKFC/Ahx AaUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=o8Dt61YLsH4S9hFNBao/bYKLJAxOvyBy3+9UBrERRWI=; b=YDTVq4K1neN8v7QrmW6XJLv4mNwm2n8FCW6AL2eiHC49V5RaZ7FOy9NNJ3g9jfmOF7 SJlothsFqGpsTyVUmWuMojSQZXs4NZ80Rnr1o3OUopmD7dRx61SdGiLni0XrYs6I2Nfj bTjVgjrppiC2CO04cab40wvzUkP/aold/7liKBZgr7MNFjVaBSSR+QGapM0VcMz+nKWz maoDpktMAg5CFFXMDpa8i2Tiw6TvdIOSggoXb2XmShknoR0HFJSK2yVQi4hVB+N/v7Tt 2pvWZLg4ZFvwJMYtp4/3aASpwHIzGfqp+/inR6+86dyE5jByWLRcKKLi28PCac7qxCuB QO5g== X-Gm-Message-State: AE9vXwNzj9acJ3nXNVL314GP9Xmnb63yDyc5UWOIuvrIB7fDgbEKa9mOYUthNKHTwgqZew== X-Received: by 10.28.223.9 with SMTP id w9mr3796618wmg.65.1472213192186; Fri, 26 Aug 2016 05:06:32 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (ANice-651-1-19-245.w86-203.abo.wanadoo.fr. [86.203.34.245]) by smtp.gmail.com with ESMTPSA id hy3sm19730097wjb.8.2016.08.26.05.06.31 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 26 Aug 2016 05:06:31 -0700 (PDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: [ZFS] ARC accounting bug ? From: Ben RUBSON In-Reply-To: <067ac0ed-1f2d-c9fb-de4d-ed5da62fe10c@internetx.com> Date: Fri, 26 Aug 2016 14:06:30 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <9EAAEEC5-0FFF-474D-9AD0-4447CD6B549B@gmail.com> References: <5B8C6663-5D21-4C03-AC69-CB7E70D3A6C2@gmail.com> <067ac0ed-1f2d-c9fb-de4d-ed5da62fe10c@internetx.com> To: FreeBSD FS X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2016 12:06:34 -0000 Juergen, thank you for your answer. So where do you think these ARC misses come from ? I can't find out :S Thank you ! > On 26 Aug 2016, at 14:01, InterNetX - Juergen Gotteswinter = wrote: >=20 > no bug, everything like i whould expect >=20 > Am 26.08.2016 um 11:40 schrieb Ben RUBSON: >> uname -v >> FreeBSD 11.0-RC2 #0 r304729: Wed Aug 24 06:59:03 UTC 2016 = root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>=20 From owner-freebsd-fs@freebsd.org Fri Aug 26 12:16:29 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7506BB700BC for ; Fri, 26 Aug 2016 12:16:29 +0000 (UTC) (envelope-from juergen.gotteswinter@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) by mx1.freebsd.org (Postfix) with ESMTP id 3B3C6896 for ; Fri, 26 Aug 2016 12:16:29 +0000 (UTC) (envelope-from juergen.gotteswinter@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id 2EDEC45FC0FF; Fri, 26 Aug 2016 14:16:27 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vumoIX+8Z91S; Fri, 26 Aug 2016 14:16:22 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id DF1024C4C85A; Fri, 26 Aug 2016 14:16:22 +0200 (CEST) Reply-To: juergen.gotteswinter@internetx.com Subject: Re: [ZFS] ARC accounting bug ? References: <5B8C6663-5D21-4C03-AC69-CB7E70D3A6C2@gmail.com> <067ac0ed-1f2d-c9fb-de4d-ed5da62fe10c@internetx.com> <9EAAEEC5-0FFF-474D-9AD0-4447CD6B549B@gmail.com> To: Ben RUBSON , FreeBSD FS From: InterNetX - Juergen Gotteswinter Organization: InterNetX GmbH Message-ID: Date: Fri, 26 Aug 2016 14:16:21 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <9EAAEEC5-0FFF-474D-9AD0-4447CD6B549B@gmail.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2016 12:16:29 -0000 so, you know the difference between - metadata - data ? if yes, what is your find actually doing in this case? regarding arc http://dtrace.org/blogs/brendan/2012/01/09/activity-of-the-zfs-arc/ https://www.patpro.net/blog/index.php/2014/03/19/2628-zfs-primarycache-all-versus-metadata/ Am 26.08.2016 um 14:06 schrieb Ben RUBSON: > Juergen, thank you for your answer. > So where do you think these ARC misses come from ? I can't find out :S > Thank you ! > >> On 26 Aug 2016, at 14:01, InterNetX - Juergen Gotteswinter wrote: >> >> no bug, everything like i whould expect >> >> Am 26.08.2016 um 11:40 schrieb Ben RUBSON: >>> uname -v >>> FreeBSD 11.0-RC2 #0 r304729: Wed Aug 24 06:59:03 UTC 2016 root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>> > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Fri Aug 26 12:28:57 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0596DB70330 for ; Fri, 26 Aug 2016 12:28:57 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x232.google.com (mail-wm0-x232.google.com [IPv6:2a00:1450:400c:c09::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F0CED51 for ; Fri, 26 Aug 2016 12:28:56 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x232.google.com with SMTP id o80so116950825wme.1 for ; Fri, 26 Aug 2016 05:28:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=F3jApljpOWP8y/fi8jJ1fuZ3JXVzAJchoDMMYCjmrM4=; b=PUNonFeg8rs3wQeg9DEClQ9dfrt9DtAyHmmXs6h29dGKI8nEdHFQr9v07dnaYo907O /A/1C0mNE3R0rp23T+LP8J66S/q+rMqnBxG4mGCPBKsknN/qusPAHuLFPu3xBxd5pASD Ykw3lbFU4zad+FDY77jDJzqkFfLABY84JTvjuG5U0x1xi0NZhC6+xNQl7Wvsj26zkLuN uqLyZSY9K2a8hpArzC0eo8X25Spo6usVxQnQNP/MePRAu8mW9FpwP0Qw4efLR53+GBA3 v7Y+OcPKsqbsQqCQv6NinuJZtCSsJDsWbwrFeVEyGmnLL3P/TarWe1sriacp4sHQdbJN Wi/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=F3jApljpOWP8y/fi8jJ1fuZ3JXVzAJchoDMMYCjmrM4=; b=Vs6uz9Kvo6PRP/t0KjpTztF3CZQbyT97gPew/WfwyR+ZftHctxbyPKyv/pt825M1o7 75i1WNZJ/xvChSq5gk6V384yojdQfewRbR3pnyHF3vpRU8gnZ3BnSSB5Dj7fBfHG7WER vmggss4UuQ73nDVRbf8wO1pfoznacOAGTKjqkzPIJoXB1IWD5YYAsv9cnp05gKzvSqN3 RpbhXws0z+9i3vdJaNo14Wzw5Z1Clni4mHDpn7vx3IPHFHEp2J4f02cP78of5sCxc8mM /r40saVsPgYjRr5tBqS5u+U2WKpRbR4rUY0R0/0TpxOIY0LuX8O1BAsyQ/oKCTUHBC7t x2nQ== X-Gm-Message-State: AE9vXwOgBvLxwtnEnaita7uRQ5mwZByK++EBDp67Sd/GmM4NxnJeThKrPYvGSFuM/xDxCg== X-Received: by 10.194.175.170 with SMTP id cb10mr3994668wjc.17.1472214534625; Fri, 26 Aug 2016 05:28:54 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (ANice-651-1-19-245.w86-203.abo.wanadoo.fr. [86.203.34.245]) by smtp.gmail.com with ESMTPSA id ub8sm19821922wjc.39.2016.08.26.05.28.53 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 26 Aug 2016 05:28:54 -0700 (PDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: [ZFS] ARC accounting bug ? From: Ben RUBSON In-Reply-To: Date: Fri, 26 Aug 2016 14:28:52 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <91665D7B-D2EA-4A7F-BE8D-67BE238DAF6B@gmail.com> References: <5B8C6663-5D21-4C03-AC69-CB7E70D3A6C2@gmail.com> <067ac0ed-1f2d-c9fb-de4d-ed5da62fe10c@internetx.com> <9EAAEEC5-0FFF-474D-9AD0-4447CD6B549B@gmail.com> To: FreeBSD FS X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2016 12:28:57 -0000 Yes Juergen, metadata only stores in ARC files' info (location in pool = and some other useful info), not the user data. I think "find" only needs metadata, as it does not read the files' = content. Which can be "verified" because : - there is no IO at all on the pool (after the first find loop, once ARC = has the needed metadata) ; - there is no ARC demand data at all. But I may be wrong, so of course feel free to correct me. Thank you. Thank you for the links, I already read them before posting here, but I = may have missed something. Ben > On 26 Aug 2016, at 14:16, InterNetX - Juergen Gotteswinter = wrote: >=20 > so, you know the difference between >=20 > - metadata > - data >=20 > ? >=20 > if yes, what is your find actually doing in this case? >=20 > regarding arc >=20 > http://dtrace.org/blogs/brendan/2012/01/09/activity-of-the-zfs-arc/ > = https://www.patpro.net/blog/index.php/2014/03/19/2628-zfs-primarycache-all= -versus-metadata/ >=20 >=20 >=20 > Am 26.08.2016 um 14:06 schrieb Ben RUBSON: >> Juergen, thank you for your answer. >> So where do you think these ARC misses come from ? I can't find out = :S >> Thank you ! >>=20 >>> On 26 Aug 2016, at 14:01, InterNetX - Juergen Gotteswinter = wrote: >>>=20 >>> no bug, everything like i whould expect >>>=20 >>> Am 26.08.2016 um 11:40 schrieb Ben RUBSON: >>>> uname -v >>>> FreeBSD 11.0-RC2 #0 r304729: Wed Aug 24 06:59:03 UTC 2016 = root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC >>>> _______________________________________________ >>>> freebsd-fs@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>> To unsubscribe, send any mail to = "freebsd-fs-unsubscribe@freebsd.org" >>>>=20 >>=20 >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>=20 From owner-freebsd-fs@freebsd.org Fri Aug 26 15:25:41 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 23729B70A79 for ; Fri, 26 Aug 2016 15:25:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 086EAE42 for ; Fri, 26 Aug 2016 15:25:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u7QFPe0l039421 for ; Fri, 26 Aug 2016 15:25:40 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 212168] [panic] [UFS] use-after-free panic (0xdeadc0dedeadc0de) Date: Fri, 26 Aug 2016 15:25:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-RC1 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2016 15:25:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D212168 Kirk McKusick changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |Open --- Comment #1 from Kirk McKusick --- From: Konstantin Belousov Date: Fri, 26 Aug 2016 13:51:00 +0300 To: Kirk McKusick Subject: Re: [Bug 212168] [panic] [UFS] use-after-free panic (0xdeadc0dedeadc0de) Cc: gjb@freebsd.org On Thu, Aug 25, 2016 at 10:56:20PM -0700, Kirk McKusick wrote: >> From: bugzilla-noreply@freebsd.org >> Date: Fri, 26 Aug 2016 05:19:37 +0000 >> To: freebsd-fs@FreeBSD.org >> Subject: [Bug 212168] [panic] [UFS] use-after-free panic (0xdeadc0dedead= c0de) >> >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D212168 >> >> Glen Barber changed: >> >> What |Removed |Added >> ------------------------------------------------------------------------= ---- >> Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org > > My first guess is that a buffer is being allocated and is not initializing > the bp->b_dep list. Any thoughts? Which buffer would it be ? Note that buf_alloc() unconditionally performs LIST_INIT(&bp->b_dep). More, buf_free() calls buf_deallocate() for non-empty b_dep, which must panic much earlier. I can only think of pbuf which has dandling b_dep for your hypothesis, but then I do not understand how the dependencies could get attached to it, even if the current buffer is cluster. I put a trivial patch below to be extra careful with b_dep for pbufs. Please try it. But really I do think that this is for normal buffer and the issue is somewhere else. Of course, the fact that intensive testing on x86 and somewhat less extensive testing on ARM and other platform did not show anything similar makes the arm64 specific bug much more feasible theory. diff --git a/sys/vm/vm_pager.c b/sys/vm/vm_pager.c index fccd1c8..3e6c447 100644 --- a/sys/vm/vm_pager.c +++ b/sys/vm/vm_pager.c @@ -375,6 +375,7 @@ initpbuf(struct buf *bp) bp->b_ioflags =3D 0; bp->b_iodone =3D NULL; bp->b_error =3D 0; + LIST_INIT(&bp->b_dep); BUF_LOCK(bp, LK_EXCLUSIVE, NULL); } @@ -472,6 +473,7 @@ relpbuf(struct buf *bp, int *pfreecnt) KASSERT(bp->b_vp =3D=3D NULL, ("relpbuf with vp")); KASSERT(bp->b_bufobj =3D=3D NULL, ("relpbuf with bufobj")); + KASSERT(LIST_EMPTY(&bp->b_dep), ("relpbuf with deps")); BUF_UNLOCK(bp); --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Aug 26 15:27:23 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C8082B70AE3 for ; Fri, 26 Aug 2016 15:27:23 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x22c.google.com (mail-wm0-x22c.google.com [IPv6:2a00:1450:400c:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5A0DEED3 for ; Fri, 26 Aug 2016 15:27:23 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x22c.google.com with SMTP id i5so124771576wmg.0 for ; Fri, 26 Aug 2016 08:27:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=N9UIWd19qYFtJWZ/ivHMEkKndLfUD3I+mgnGCWLCLB8=; b=Nw/oSxLe15lPNNaVQL5j8xBB7E4KGXZO/LpvkBEiE8/yQlyJok0DY+QaR8mH2GYmit hhZr2lMLCxUEwBROKgtAEch4CsukvObcANNXnk9gw1t4cb7VESLTEqAiVgevkA8Zkc3c DGLcd65S27o9Dk4ZorsLe2y+1qCcEldKzowvCS2Rl5oH6oAgxvAC9LdgQnIyDieGhruW aY6JGpNCgSyY1rGDRF/s5fJOwvoj9mffgPfwmflyw4A5jQ+Cs0G+jKbB0xzdsDhXRDxc 4dTflColNB8+hHUqxOj8nbGx7PStxHLK/m7ZyRG2wcjCjQ6VRyfrIUBOH5ZtSOHInD/e E28Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=N9UIWd19qYFtJWZ/ivHMEkKndLfUD3I+mgnGCWLCLB8=; b=XTdW8ANl/22IrBLqmGrxdwx+o4Hlk6Y5fBYZvQd+2dycd4yl/bBa0kpJViWprYJiXI 66qaL8aAzGqyuyR3Q5535wPpDCGwqTG8TfKLHRP4iIMIbPTvTYrXMyJho5kH0ifOiVW/ IV32+vpH1sXnpsv1LXj2zOHQZePnA5jpfjenZ+kxxQq0XbAROnmv6hSre66UFPJgNy1g wpQwGLmFq6rDLVrT7nbnFMHNnIhOTNOUQcPOusVj/C30nxiv9P9Ioe77JqaEXt7xuHVd DGphsOIjDGX5hBH0qNg60WcQ2yMFOacYx88EMlkyqC+jMEF7EdYSIm01yvL2PUahgEmJ Vegw== X-Gm-Message-State: AE9vXwOQYCLR+XPujr8od7OFpdt3qyEYf6LnFOh64X0+g1N1//M+UELbxzY2Y94rm1Xtmw== X-Received: by 10.194.187.134 with SMTP id fs6mr4660895wjc.3.1472225241302; Fri, 26 Aug 2016 08:27:21 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (ANice-651-1-19-245.w86-203.abo.wanadoo.fr. [86.203.34.245]) by smtp.gmail.com with ESMTPSA id e5sm44129665wma.13.2016.08.26.08.27.20 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 26 Aug 2016 08:27:20 -0700 (PDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: [ZFS] ARC accounting bug ? From: Ben RUBSON In-Reply-To: <91665D7B-D2EA-4A7F-BE8D-67BE238DAF6B@gmail.com> Date: Fri, 26 Aug 2016 17:27:19 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <9A3B2699-B077-4F72-90A5-63D7E22BE652@gmail.com> References: <5B8C6663-5D21-4C03-AC69-CB7E70D3A6C2@gmail.com> <067ac0ed-1f2d-c9fb-de4d-ed5da62fe10c@internetx.com> <9EAAEEC5-0FFF-474D-9AD0-4447CD6B549B@gmail.com> <91665D7B-D2EA-4A7F-BE8D-67BE238DAF6B@gmail.com> To: FreeBSD FS X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2016 15:27:23 -0000 I'm sorry but I still can't find out where do these ARC demand metadata = misses come from. In addition, why they are not cached, as my ARC is 1% used. And why I don't see any disk activity to honour them. I would really be glad if you could enlighten me :) Many thanks again, Ben > On 26 Aug 2016, at 14:28, Ben RUBSON wrote: >=20 > Yes Juergen, metadata only stores in ARC files' info (location in pool = and some other useful info), not the user data. >=20 > I think "find" only needs metadata, as it does not read the files' = content. > Which can be "verified" because : > - there is no IO at all on the pool (after the first find loop, once = ARC has the needed metadata) ; > - there is no ARC demand data at all. > But I may be wrong, so of course feel free to correct me. Thank you. >=20 > Thank you for the links, I already read them before posting here, but = I may have missed something. >=20 > Ben >=20 >> On 26 Aug 2016, at 14:16, InterNetX - Juergen Gotteswinter = wrote: >>=20 >> so, you know the difference between >>=20 >> - metadata >> - data >>=20 >> ? >>=20 >> if yes, what is your find actually doing in this case? >>=20 >> regarding arc >>=20 >> http://dtrace.org/blogs/brendan/2012/01/09/activity-of-the-zfs-arc/ >> = https://www.patpro.net/blog/index.php/2014/03/19/2628-zfs-primarycache-all= -versus-metadata/ >>=20 >>=20 >>=20 >> Am 26.08.2016 um 14:06 schrieb Ben RUBSON: >>> Juergen, thank you for your answer. >>> So where do you think these ARC misses come from ? I can't find out = :S >>> Thank you ! >>>=20 >>>> On 26 Aug 2016, at 14:01, InterNetX - Juergen Gotteswinter = wrote: >>>>=20 >>>> no bug, everything like i whould expect >>>>=20 >>>> Am 26.08.2016 um 11:40 schrieb Ben RUBSON: >>>>> uname -v >>>>> FreeBSD 11.0-RC2 #0 r304729: Wed Aug 24 06:59:03 UTC 2016 = root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC From owner-freebsd-fs@freebsd.org Fri Aug 26 15:46:20 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7646DB70F36 for ; Fri, 26 Aug 2016 15:46:20 +0000 (UTC) (envelope-from lkateley@kateley.com) Received: from mail-it0-x230.google.com (mail-it0-x230.google.com [IPv6:2607:f8b0:4001:c0b::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3C1BEB74 for ; Fri, 26 Aug 2016 15:46:20 +0000 (UTC) (envelope-from lkateley@kateley.com) Received: by mail-it0-x230.google.com with SMTP id x131so338696824ite.0 for ; Fri, 26 Aug 2016 08:46:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kateley-com.20150623.gappssmtp.com; s=20150623; h=reply-to:subject:references:to:from:organization:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=plt6XKBZqbOD4MTUY1dEj1hsEPJSvZRPWAIR2g9PPYA=; b=jL2Jg3Nq8sdV6AljTPXvU2BX/OkM1mGdizvn/Fi89DzAkOxWMhytjPQaJsF+ztWi3Q eLQc0b6HLh3w/TCuaWs+6KO8nfQtO0I96teU4XhNYjai4i7Uo9rXCEa+S0ocGhj4i77U KmVH13VsnzckEOJpryEfWqxUgqmRbZa0MYqBqx6l1kVucv9NBNbiZOjPFXt1EDOcn/bS 4zx43LrEPTN0NsgZ4nbWH5iYnNWqi9o/B4xrUiNeJN2dxnZ6AZZcmKB5bwDh6yigD/AW R4AHPiuPW4dHUTiRoYdU3sJKWJknQNXyXFFhNLo25vFA4wCTRgglwQx3B2tVL/TtMgs7 vySg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:reply-to:subject:references:to:from:organization :message-id:date:user-agent:mime-version:in-reply-to :content-transfer-encoding; bh=plt6XKBZqbOD4MTUY1dEj1hsEPJSvZRPWAIR2g9PPYA=; b=TmY5Cf80qMDnsYagRajf1kGYh6ah60dMC+5An5e+x6XBZQ7iBfOorkc6kqAFX0s7KH k818AXg5w2dbSMs2VNWq6jMkiyr7qnilgMX49jeoqubroIIlG2h8ksgid9wrtVbYk5YM dxvdhudySLy9ZVDGTRuYm3NTb07sJNlxBrTCUGr2fkmE5gWAN/3KT0TyOReuk6cTyn7G HXvO41BV0Z9nw7Dh6Cco5LBzVpfRRb7f5BWWSiP413ibuBWO0dgBRSYbYSgygwY1ww1o 6RFuAaGEtx9XMlKhalAyCe8LKp18z/Rx32HdvTznIBE5q/SO8wrRAqPWBDyoVqUlRUW8 njkQ== X-Gm-Message-State: AE9vXwMihtAmaYnoBQhv1fyk3PWTOKjylzCLLgM+7z6urpD93liNuMOYbJY4y8c58AaeXQ== X-Received: by 10.107.9.42 with SMTP id j42mr4766488ioi.33.1472226379092; Fri, 26 Aug 2016 08:46:19 -0700 (PDT) Received: from Kateleyco-iMac.local (c-50-188-36-30.hsd1.mn.comcast.net. [50.188.36.30]) by smtp.googlemail.com with ESMTPSA id i80sm7881791ita.5.2016.08.26.08.46.18 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 26 Aug 2016 08:46:18 -0700 (PDT) Reply-To: linda@kateley.com Subject: Re: [ZFS] ARC accounting bug ? References: <5B8C6663-5D21-4C03-AC69-CB7E70D3A6C2@gmail.com> <067ac0ed-1f2d-c9fb-de4d-ed5da62fe10c@internetx.com> <9EAAEEC5-0FFF-474D-9AD0-4447CD6B549B@gmail.com> <91665D7B-D2EA-4A7F-BE8D-67BE238DAF6B@gmail.com> <9A3B2699-B077-4F72-90A5-63D7E22BE652@gmail.com> To: freebsd-fs@freebsd.org From: Linda Kateley Organization: Kateley Company Message-ID: <453b967d-4139-c6b6-a27a-223ed4d92c74@kateley.com> Date: Fri, 26 Aug 2016 10:46:17 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <9A3B2699-B077-4F72-90A5-63D7E22BE652@gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2016 15:46:20 -0000 Metadata updates when you access a file. Atime is most often updated and you will miss. If you are running vm's on zfs I always recommend to turn off atime updates. Metadata is usually tiny so you probably won't see disk activity On 8/26/16 10:27 AM, Ben RUBSON wrote: > I'm sorry but I still can't find out where do these ARC demand metadata misses come from. > In addition, why they are not cached, as my ARC is 1% used. > And why I don't see any disk activity to honour them. > > I would really be glad if you could enlighten me :) > > Many thanks again, > > Ben > > > >> On 26 Aug 2016, at 14:28, Ben RUBSON wrote: >> >> Yes Juergen, metadata only stores in ARC files' info (location in pool and some other useful info), not the user data. >> >> I think "find" only needs metadata, as it does not read the files' content. >> Which can be "verified" because : >> - there is no IO at all on the pool (after the first find loop, once ARC has the needed metadata) ; >> - there is no ARC demand data at all. >> But I may be wrong, so of course feel free to correct me. Thank you. >> >> Thank you for the links, I already read them before posting here, but I may have missed something. >> >> Ben >> >>> On 26 Aug 2016, at 14:16, InterNetX - Juergen Gotteswinter wrote: >>> >>> so, you know the difference between >>> >>> - metadata >>> - data >>> >>> ? >>> >>> if yes, what is your find actually doing in this case? >>> >>> regarding arc >>> >>> http://dtrace.org/blogs/brendan/2012/01/09/activity-of-the-zfs-arc/ >>> https://www.patpro.net/blog/index.php/2014/03/19/2628-zfs-primarycache-all-versus-metadata/ >>> >>> >>> >>> Am 26.08.2016 um 14:06 schrieb Ben RUBSON: >>>> Juergen, thank you for your answer. >>>> So where do you think these ARC misses come from ? I can't find out :S >>>> Thank you ! >>>> >>>>> On 26 Aug 2016, at 14:01, InterNetX - Juergen Gotteswinter wrote: >>>>> >>>>> no bug, everything like i whould expect >>>>> >>>>> Am 26.08.2016 um 11:40 schrieb Ben RUBSON: >>>>>> uname -v >>>>>> FreeBSD 11.0-RC2 #0 r304729: Wed Aug 24 06:59:03 UTC 2016 root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Fri Aug 26 15:48:53 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44D55B70FA2 for ; Fri, 26 Aug 2016 15:48:53 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x232.google.com (mail-wm0-x232.google.com [IPv6:2a00:1450:400c:c09::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9FABC10 for ; Fri, 26 Aug 2016 15:48:52 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x232.google.com with SMTP id q128so284659976wma.1 for ; Fri, 26 Aug 2016 08:48:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=V8gruszECuHkNICP5A7V68TwdukGZC5YwKvzek4mh8E=; b=s+j1cIVMHLCJ3riZgZY2CQDYGOjPxQOYantF9WpMnNQGqHbwHu3PoBzH1qj3uO0Azh hxQPAgYJTcIM+g7e6SA3Uc/mjVSUuOx4sRpypkIimP/L5+gSJ37s6tHcDaPWkB5hPGI5 qw/LU+g6asHiPmA1Y3BF9bVUVlRgY0HPc/9ucpWAFLJxfVlo1CguMSxoQYxmY6x+K/IQ BgiwVXryqOJsMZ9GcAkCppaJJS0C0pl0kHxWL0bJ3a8DiOd1UgcEeaf5o3GtD3dTikM/ qclPa27IzZoCACGazjSG7GonN1TqkyI4JXg4P2Q7LZFhfVv5UGejbPFmjEuhdspVxGMu vLmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=V8gruszECuHkNICP5A7V68TwdukGZC5YwKvzek4mh8E=; b=O9WgmTT2NUm6KALozmzRkEfn5Mky1s5GYeLTws19wxp0IGqeA32/wuebZVR80NVV7V 24HMPf1M1BkrSHn3D73G9oCLv3P0p0CpuCIBkuhz/zNYCgpZBpW709OetwkGU3mAKKbd gnErQgJb2an038+ZsXZsa0oWTQkTf6X9VI+94+rMMevCwqecvqBR9GldSA6uJxdnmrCM VKj7oT+kujhwXaab7fAn7C+HDvBqo3l+psU8104Y67+N/rJQGSsGqlnotkaeBhrp+ceT Imgz6zUaEx+6S4TU93dOlL6uZmUUoBytcPPn5EKHM42ggUZC2ObGFRgOXgaJ8u84OB+A XWrg== X-Gm-Message-State: AE9vXwODLNP6LkXpD++tjo+XfCV0HkMLeiplAQBtDjOe6NX9qC9lnDOLefNFlbyczP2OPA== X-Received: by 10.194.60.2 with SMTP id d2mr5010239wjr.179.1472226531013; Fri, 26 Aug 2016 08:48:51 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (ANice-651-1-19-245.w86-203.abo.wanadoo.fr. [86.203.34.245]) by smtp.gmail.com with ESMTPSA id o142sm44307494wme.20.2016.08.26.08.48.50 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 26 Aug 2016 08:48:50 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: [ZFS] ARC accounting bug ? From: Ben RUBSON In-Reply-To: <453b967d-4139-c6b6-a27a-223ed4d92c74@kateley.com> Date: Fri, 26 Aug 2016 17:48:49 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5B8C6663-5D21-4C03-AC69-CB7E70D3A6C2@gmail.com> <067ac0ed-1f2d-c9fb-de4d-ed5da62fe10c@internetx.com> <9EAAEEC5-0FFF-474D-9AD0-4447CD6B549B@gmail.com> <91665D7B-D2EA-4A7F-BE8D-67BE238DAF6B@gmail.com> <9A3B2699-B077-4F72-90A5-63D7E22BE652@gmail.com> <453b967d-4139-c6b6-a27a-223ed4d92c74@kateley.com> To: FreeBSD FS X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2016 15:48:53 -0000 Linda, Thank you very much for your suggestion. However atime is off on my pool. > On 26 Aug 2016, at 17:46, Linda Kateley wrote: >=20 > Metadata updates when you access a file. Atime is most often updated = and you will miss. >=20 > If you are running vm's on zfs I always recommend to turn off atime = updates. >=20 > Metadata is usually tiny so you probably won't see disk activity >=20 >=20 > On 8/26/16 10:27 AM, Ben RUBSON wrote: >> I'm sorry but I still can't find out where do these ARC demand = metadata misses come from. >> In addition, why they are not cached, as my ARC is 1% used. >> And why I don't see any disk activity to honour them. >>=20 >> I would really be glad if you could enlighten me :) >>=20 >> Many thanks again, >>=20 >> Ben >>=20 >>=20 >>=20 >>> On 26 Aug 2016, at 14:28, Ben RUBSON wrote: >>>=20 >>> Yes Juergen, metadata only stores in ARC files' info (location in = pool and some other useful info), not the user data. >>>=20 >>> I think "find" only needs metadata, as it does not read the files' = content. >>> Which can be "verified" because : >>> - there is no IO at all on the pool (after the first find loop, once = ARC has the needed metadata) ; >>> - there is no ARC demand data at all. >>> But I may be wrong, so of course feel free to correct me. Thank you. >>>=20 >>> Thank you for the links, I already read them before posting here, = but I may have missed something. >>>=20 >>> Ben >>>=20 >>>> On 26 Aug 2016, at 14:16, InterNetX - Juergen Gotteswinter = wrote: >>>>=20 >>>> so, you know the difference between >>>>=20 >>>> - metadata >>>> - data >>>>=20 >>>> ? >>>>=20 >>>> if yes, what is your find actually doing in this case? >>>>=20 >>>> regarding arc >>>>=20 >>>> http://dtrace.org/blogs/brendan/2012/01/09/activity-of-the-zfs-arc/ >>>> = https://www.patpro.net/blog/index.php/2014/03/19/2628-zfs-primarycache-all= -versus-metadata/ >>>>=20 >>>>=20 >>>>=20 >>>> Am 26.08.2016 um 14:06 schrieb Ben RUBSON: >>>>> Juergen, thank you for your answer. >>>>> So where do you think these ARC misses come from ? I can't find = out :S >>>>> Thank you ! >>>>>=20 >>>>>> On 26 Aug 2016, at 14:01, InterNetX - Juergen Gotteswinter = wrote: >>>>>>=20 >>>>>> no bug, everything like i whould expect >>>>>>=20 >>>>>> Am 26.08.2016 um 11:40 schrieb Ben RUBSON: >>>>>>> uname -v >>>>>>> FreeBSD 11.0-RC2 #0 r304729: Wed Aug 24 06:59:03 UTC 2016 = root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Fri Aug 26 19:00:59 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E65B4A94C15 for ; Fri, 26 Aug 2016 19:00:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D30BF81 for ; Fri, 26 Aug 2016 19:00:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u7QJ0xik066721 for ; Fri, 26 Aug 2016 19:00:59 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 212168] [panic] [UFS] use-after-free panic (0xdeadc0dedeadc0de) Date: Fri, 26 Aug 2016 19:00:59 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-RC1 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: gjb@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2016 19:01:00 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D212168 --- Comment #2 from Glen Barber --- I will try this patch on the second of two machines. On the first, I have turned off SU+J for now, as well as WITNESS/INVARIANTS/etc., as there is a = time constraint on package availability. We do have one machine with which testing can be done to diagnose this more= , as I have seen the same issue on both of two of these. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Aug 27 05:27:46 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0F4CB76013 for ; Sat, 27 Aug 2016 05:27:46 +0000 (UTC) (envelope-from FreeBSD@shaneware.biz) Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by mx1.freebsd.org (Postfix) with ESMTP id 42926F05 for ; Sat, 27 Aug 2016 05:27:45 +0000 (UTC) (envelope-from FreeBSD@shaneware.biz) Received: from ppp14-2-4-72.lns21.adl2.internode.on.net (HELO leader.local) ([14.2.4.72]) by ipmail05.adl6.internode.on.net with ESMTP; 27 Aug 2016 14:52:33 +0930 Subject: Re: [ZFS] ARC accounting bug ? To: Ben RUBSON , FreeBSD FS References: From: Shane Ambler Message-ID: Date: Sat, 27 Aug 2016 14:52:32 +0930 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2016 05:27:46 -0000 On 26/08/2016 19:09, Ben RUBSON wrote: > Hello, > > Before opening a bug report, I would like to know whether what I see is "normal" or not, and why. > ### Test : > > # zfs import mypool > # zfs set primarycache=metadata mypool Well that sets the primarycache for the pool and all subsets that inherit the property. Do any sub filesystems have local settings? zfs get -r primarycache mypool And mypool is the only zpool on the machine? > # while [ 1 ]; do find /mypool/ >/dev/null; done > # zfs-mon -a > > ZFS real-time cache activity monitor > Seconds elapsed: 162 > > Cache hits and misses: > 1s 10s 60s tot > ARC hits: 79228 76030 73865 74953 > ARC misses: 22510 22184 21647 21955 > ARC demand data hits: 0 0 0 0 > ARC demand data misses: 4 7 8 7 > ARC demand metadata hits: 79230 76030 73865 74953 > ARC demand metadata misses: 22506 22177 21639 21948 > ZFETCH hits: 47 29 32 31 > ZFETCH misses:101669 98138 95433 96830 > > Cache efficiency percentage: > 10s 60s tot > ARC: 77.41 77.34 77.34 > ARC demand data: 0.00 0.00 0.00 > ARC demand metadata: 77.42 77.34 77.35 > ZFETCH: 0.03 0.03 0.03 > > ### Question : > > I don't understand why I have so many ARC misses. There is no other > activity on the server (as soon as I stop the find loop, no more ARC > hits). As soon as the first find loop is done, there is no more disk > activity (according to zpool instate -v 1), no read/write operations > on mypool. > So I'm pretty sure all metadata comes from ARC. > So why are there so many ARC misses ? Running zfs-mon on my desktop, I seem to get similar results. What I am seeing leads me to think that not all metadata is cached, maybe filename isn't cached, which can be a large string. while [ 1 ]; do find /usr/ports > /dev/null; done will list the path to every file and I see about 2 hits to a miss, yet while [ 1 ]; do ls -lR /usr/ports > /dev/null; done lists every filename as well as it's size, mod date, owner, permissions and it sits closer to 4 hits to every miss. And if the system disk cache contains the filenames that zfs isn't caching we won't need disk access to get the zfs misses. -- FreeBSD - the place to B...Storing Data Shane Ambler From owner-freebsd-fs@freebsd.org Sat Aug 27 11:07:35 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2C3AB769B7 for ; Sat, 27 Aug 2016 11:07:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E2BA1AF2 for ; Sat, 27 Aug 2016 11:07:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u7RB7ZqL041828 for ; Sat, 27 Aug 2016 11:07:35 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 212168] [panic] [UFS] use-after-free panic (0xdeadc0dedeadc0de) Date: Sat, 27 Aug 2016 11:07:35 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-RC1 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: Andrew@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2016 11:07:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D212168 Andrew Turner changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |Andrew@FreeBSD.org --- Comment #3 from Andrew Turner --- Can you get the backtrace and ps output from ddb when this happens? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Aug 27 16:15:22 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4FCE3B76E04 for ; Sat, 27 Aug 2016 16:15:22 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x22e.google.com (mail-wm0-x22e.google.com [IPv6:2a00:1450:400c:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D855D636 for ; Sat, 27 Aug 2016 16:15:21 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x22e.google.com with SMTP id i5so32301302wmg.0 for ; Sat, 27 Aug 2016 09:15:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=53D2hOiQwnhOFAzDR7gRa3I8GDHsCCCfM05QaLShoc0=; b=fKzXH5gA9UTfw6rQLrumlqQNSOQMRE3yfTOlgLPE8uBNdkOhiIF05TNS52EfRfV/Y2 89VQZmaqHQKlbySOhQfec6VxPwnU7JaPn0eR9XizKgKbqtsnu52tdDMQCpm50vmzts43 VLJZOcWpkKyPgTH88eM5YQp9QWblmuS8Eu1LaQRGSf29e9TioDowS6WZyHTZSuWMo7qs vIJW71M6OGrCQ0kQNmNKSpfJRq0YH3gjD9786ZqdhgAEJU5H51Mf7xZnIUDgRdf2zC0l qJIIDbVO4tbIGpkIK+qBRqFg9WDS7nZ58G+PKI3vqDZdHSVroYdw1CO+SsCN+r173nv/ lqeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=53D2hOiQwnhOFAzDR7gRa3I8GDHsCCCfM05QaLShoc0=; b=WoiWLijTNEFwq6Zv+gSYB6sk7xFVjphZ8L19QrlSsiVB+1BnMTZBz3ggDjiQGq8GM/ O4TlfBAAwIoM/i8IQzj8ye60f0kzobek24UvlVTwwR1vRg2UZLKNDAOnXUg6uKoh4eU6 tm1UcOGdFwXX25H4fm1Mgrw70doLEcdPfGGzOQWDp3DCW9Xh+EmoE3gx00jq9n7yvl8P HYu9tiaWXh950DlnzIvuIbX5XkTiiinLyUMUk9FJv2Tk4wCos9zk5zslxprPpTWsyUqM C5+UPbLK3DbhF723q0J5OWARPTXfzyFzdYm9bAm2kGeHWLCm63xnhTxmTVmifCai2Lnq C5ZQ== X-Gm-Message-State: AE9vXwPh5uyJjm44QpKKTFvk+IWh0KTadP/sCXHwCcEnEXHjlaio3iofWR8IcGsBEdZRiQ== X-Received: by 10.28.197.133 with SMTP id v127mr3702429wmf.26.1472314519851; Sat, 27 Aug 2016 09:15:19 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (ANice-651-1-19-245.w86-203.abo.wanadoo.fr. [86.203.34.245]) by smtp.gmail.com with ESMTPSA id n2sm25327205wjd.1.2016.08.27.09.15.18 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 27 Aug 2016 09:15:19 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: [ZFS] ARC accounting bug ? From: Ben RUBSON In-Reply-To: Date: Sat, 27 Aug 2016 18:15:18 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <71DED907-10BE-44C2-982B-12974152895D@gmail.com> References: To: FreeBSD FS X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2016 16:15:22 -0000 > On 27 Aug 2016, at 07:22, Shane Ambler wrote: >=20 > On 26/08/2016 19:09, Ben RUBSON wrote: >> Hello, >>=20 >> Before opening a bug report, I would like to know whether what I see >> is "normal" or not, and why. >>=20 >> ### Test : >>=20 >> # zfs import mypool >> # zfs set primarycache=3Dmetadata mypool >=20 > Well that sets the primarycache for the pool and all subsets that > inherit the property. Do any sub filesystems have local settings? No. > zfs get -r primary cache mypool >=20 > And mypool is the only zpool on the machine? Yes. >> # while [ 1 ]; do find /mypool/ >/dev/null; done >>=20 >> # zfs-mon -a >>=20 >> ZFS real-time cache activity monitor >> Seconds elapsed: 162 >>=20 >> Cache hits and misses: >> 1s 10s 60s tot >> ARC hits: 79228 76030 73865 74953 >> ARC misses: 22510 22184 21647 21955 >> ARC demand data hits: 0 0 0 0 >> ARC demand data misses: 4 7 8 7 >> ARC demand metadata hits: 79230 76030 73865 74953 >> ARC demand metadata misses: 22506 22177 21639 21948 >> ZFETCH hits: 47 29 32 31 >> ZFETCH misses:101669 98138 95433 96830 >>=20 >> Cache efficiency percentage: >> 10s 60s tot >> ARC: 77.41 77.34 77.34 >> ARC demand data: 0.00 0.00 0.00 >> ARC demand metadata: 77.42 77.34 77.35 >> ZFETCH: 0.03 0.03 0.03 >>=20 >> ### Question : >>=20 >> I don't understand why I have so many ARC misses. There is no other >> activity on the server (as soon as I stop the find loop, no more ARC >> hits). As soon as the first find loop is done, there is no more disk >> activity (according to zpool iostat -v 1), no read/write operations >> on mypool. >> So I'm pretty sure all metadata comes from ARC. >> So why are there so many ARC misses ? >=20 > Running zfs-mon on my desktop, I seem to get similar results. Thank you for having tested it Shane. > What I am seeing leads me to think that not all metadata is cached, > maybe filename isn't cached, which can be a large string. >=20 > while [ 1 ]; do find /usr/ports > /dev/null; done >=20 > will list the path to every file and I see about 2 hits to a miss, yet >=20 > while [ 1 ]; do ls -lR /usr/ports > /dev/null; done >=20 > lists every filename as well as it's size, mod date, owner, = permissions > and it sits closer to 4 hits to every miss. >=20 > And if the system disk cache contains the filenames that zfs isn't = caching we won't need disk access to get the zfs misses. Playing with these commands : # dtrace -n 'sdt:zfs::arc-hit {@[execname, stack()] =3D count();}' # dtrace -n 'sdt:zfs::arc-miss {@[execname, stack()] =3D count();}' We can see that these are readdir calls which produce arc-misses, and = that readdir calls also produce arc-hits. It would be interesting to know why some lead to hits, and some lead to = misses. (note that ls -lR / rsync commands produces exactly the same dtrace = results/numbers as find command) Ben From owner-freebsd-fs@freebsd.org Sat Aug 27 19:17:36 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8377AB7739D for ; Sat, 27 Aug 2016 19:17:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4840C1DF for ; Sat, 27 Aug 2016 19:17:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u7RJHaEQ005588 for ; Sat, 27 Aug 2016 19:17:36 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 212168] [panic] [UFS] use-after-free panic (0xdeadc0dedeadc0de) Date: Sat, 27 Aug 2016 19:17:36 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-RC1 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: peter@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2016 19:17:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D212168 Peter Wemm changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |peter@FreeBSD.org --- Comment #4 from Peter Wemm --- >From an earlier console log that Glen has been capturing: Stopped at softdep_disk_io_initiation+0x50: ldr x21, [x8, #= 664] db> set $lines 0 db> bt Tracing pid 44795 tid 101060 td 0xfffffd0027a859c0 db_trace_self() at db_stack_trace+0xf0 db_stack_trace() at db_command+0x254 db_command() at db_command_loop+0x60 db_command_loop() at db_trap+0xf0 db_trap() at kdb_trap+0x180 kdb_trap() at data_abort+0x19c data_abort() at handle_el1h_sync+0x68 handle_el1h_sync() at ffs_geom_strategy+0x108 ffs_geom_strategy() at ffs_geom_strategy+0x108 ffs_geom_strategy() at bufwrite+0x210 bufwrite() at softdep_freefile+0x66c softdep_freefile() at ffs_vfree+0x28 ffs_vfree() at ufs_inactive+0x2a4 ufs_inactive() at VOP_INACTIVE_APV+0xcc VOP_INACTIVE_APV() at vinactive+0xb8 vinactive() at vputx+0x30c vputx() at vn_close+0xf0 vn_close() at vn_closefile+0x58 vn_closefile() at closef+0x1f8 closef() at fdescfree_fds+0x8c fdescfree_fds() at fdescfree+0x468 fdescfree() at exit1+0x48c exit1() at sys_sys_exit+0x10 sys_sys_exit() at do_el0_sync+0x4c0 do_el0_sync() at handle_el0_sync+0x64 handle_el0_sync() at 0x41b670 db> Every one of these I've seen has been in the exit->close->inactive->freefile->ffs_geom_strategy->softdep_disk_io_initiat= ion path, and always at that precise location. It's always a [0xdeadc0dedeadc0= de + offset] memory read. Unfortunately I can't see any ps output in the console for any of these. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Aug 27 19:46:23 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1444DB77CCB for ; Sat, 27 Aug 2016 19:46:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0404F1E7 for ; Sat, 27 Aug 2016 19:46:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u7RJkMF3067737 for ; Sat, 27 Aug 2016 19:46:22 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 212168] [panic] [UFS] use-after-free panic (0xdeadc0dedeadc0de) Date: Sat, 27 Aug 2016 19:46:23 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-RC1 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: kib@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2016 19:46:23 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D212168 Konstantin Belousov changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kib@FreeBSD.org --- Comment #5 from Konstantin Belousov --- (In reply to Peter Wemm from comment #4) There should be missed ffs_freefile() frame in the backtrace between bufwri= te() and softdep_freefile(). If I am right, the failing buffer is actually for = the inode block. --=20 You are receiving this mail because: You are the assignee for the bug.=