From nobody Fri Jun 17 22:39:56 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 72756845530 for ; Fri, 17 Jun 2022 22:40:05 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-qk1-x736.google.com (mail-qk1-x736.google.com [IPv6:2607:f8b0:4864:20::736]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LPvBD2NGHz3R3T for ; Fri, 17 Jun 2022 22:40:04 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: by mail-qk1-x736.google.com with SMTP id n197so4155186qke.1 for ; Fri, 17 Jun 2022 15:40:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:message-id:date:mime-version:user-agent:content-language:to :cc:references:from:subject:in-reply-to:content-transfer-encoding; bh=Nj99oeDfRVqDnxgXS3ZQEUPwmwWFYRbnphX9Hh0utqg=; b=Qe4GDIH2WmA92IUBMuAg3B7HOOgbLd8r0Gkh+opiNuWbMJsRccGeCeskqRlHe/x++n SoRXYNiH3/DiYl54PjOkRlufEWSpB/dysM29JrERe+6IN2x8LDGwEwunrmGSLmnzgxUs NEfAlrXP+AEoXBdhoIMslkSuMm4NW5f0oyd8GZ0NOp2Q1LkoE7wCUiay2Ey6RwH54DYB YYCKnYDcdVb7OF8yX5eq0gDUkHE0z006B7C7T5nYG/D/t7S8n0DM1dMxVa8GwF5W7gtn xlp4Eti3dP1Qg/e//ejfOY+ICMt6Mx2BIAqLMOpsNhSI5Pwb8gE0Zc8OM0NnQl/nSKyv OFug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:message-id:date:mime-version:user-agent :content-language:to:cc:references:from:subject:in-reply-to :content-transfer-encoding; bh=Nj99oeDfRVqDnxgXS3ZQEUPwmwWFYRbnphX9Hh0utqg=; b=uzn8a8jlzgXilPp3+hFq8pUcssylEvqiY+NVJmVcRNZXTRiGYNHtJYx3WHrFEbLQ0+ StXpi6/ClTpP8KU/FZGt5Lx/mglCSbKICBmT6T+sCWTpHB6aJcdIsdNQ/eLzrbLZuwK9 czvpodM53ViHkmoBY2XNcIFjYVQeciiw4EF7B8PYDGt3d01EfaqR8GdLMOIP9QPH/Lm3 FavsBO5oUc6nEU473gdGwS3yAtcVwwk2+DfA6e/9L8XjbUFo/0KD1fmxzxC1D0pKen0y QY16izoCmuqG7mz9o264smBbObuZ2l7/ck/9i0ePSLmeBIQZLRYg+F9t7EPSiTnvb7s8 CxcA== X-Gm-Message-State: AJIora+v0BwoziKJj3dMdDbLHtDbEu0+BviwCc0Bo8jbDG7f0myKDY/5 8b5oOAmKuqKwrrbgEFMePPA= X-Google-Smtp-Source: AGRyM1sjhGQhdLTxT4ES++RStTEOoattpB4s2TOl7chas0nX8AcBrzVrBfZ6XmXp6lCVQW0OP6Hf/w== X-Received: by 2002:a37:b82:0:b0:6a6:cef8:85de with SMTP id 124-20020a370b82000000b006a6cef885demr8817067qkl.651.1655505597663; Fri, 17 Jun 2022 15:39:57 -0700 (PDT) Received: from [10.231.1.66] (075-130-069-034.biz.spectrum.com. [75.130.69.34]) by smtp.gmail.com with ESMTPSA id g1-20020ac87f41000000b002f9399ccefasm5722730qtk.34.2022.06.17.15.39.56 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 17 Jun 2022 15:39:57 -0700 (PDT) Message-ID: <2bc77869-244e-e3bd-709d-f24bf96782bc@FreeBSD.org> Date: Fri, 17 Jun 2022 18:39:56 -0400 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:91.0) Gecko/20100101 Thunderbird/91.9.0 Content-Language: en-US To: Larry Rosenman Cc: Freebsd current References: <96f9a47659ac052b8d806bb9b7ab2b51@lerctr.org> <1a795a4e-2b1e-2cba-484b-52811546e277@FreeBSD.org> <97624e2a4e0887ede029e71c47bb6060@lerctr.org> <9ad99f2c-3d63-ccab-f730-e355f0297d72@FreeBSD.org> <5366dceb38e43bc5bd1e84cb90ea31ac@lerctr.org> From: Alexander Motin Subject: Re: SAS/SATA controllers: 8 port that support 8TB Drives In-Reply-To: <5366dceb38e43bc5bd1e84cb90ea31ac@lerctr.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4LPvBD2NGHz3R3T X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=Qe4GDIH2; dmarc=none; spf=pass (mx1.freebsd.org: domain of mavbsd@gmail.com designates 2607:f8b0:4864:20::736 as permitted sender) smtp.mailfrom=mavbsd@gmail.com X-Spamd-Result: default: False [-1.38 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.99)[-0.989]; FORGED_SENDER(0.30)[mav@FreeBSD.org,mavbsd@gmail.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; FROM_NEQ_ENVFROM(0.00)[mav@FreeBSD.org,mavbsd@gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[FreeBSD.org]; NEURAL_SPAM_MEDIUM(0.81)[0.807]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::736:from]; MLMMJ_DEST(0.00)[freebsd-current]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On 17.06.2022 18:29, Larry Rosenman wrote: > On 06/17/2022 5:24 pm, Alexander Motin wrote: >> On 17.06.2022 18:16, Larry Rosenman wrote: >>> On 06/17/2022 5:08 pm, Alexander Motin wrote: >>>> On 17.06.2022 11:59, Larry Rosenman wrote: >>>>> I'm looking to upgrade the controllers in my TrueNAS box to >>>>> something that will >>>>> support 8TB drives because apparently my LSI 2108 controllers do >>>>> not support 8TB drives. >>>>> >>>>> What's the communities recommendation? >>>>> needs to support SFF connectors for a total of 4 SFF connectors, as >>>>> I have 16 slots. >>>> >>>> We at iX are still using LSI/Broadcom HBAs, just moved from long >>>> discontinued mps(4) to newer mpr(4).  And I don't believe the problem >>>> is directly related to capacity.  According to my observations it may >>>> be Seagate HDDs of/above certain (8TB) generation.  We do not use >>>> Seagate HDDs in our products, so about that instability I only heard >>>> from forums and TrueNAS community user reports. >>> >>> This is a mfi(4) set of controllers, and a ST80000Nm0045 8TB (CMR) >>> drive. >>> >>> Is this a bad combo? >>> >>> mfi0: 9973 (708793330s/0x0002/WARN) - PD 00(e0xfc/s3) is not supported >>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>> (probe0:mfi0:0:0:0): Retrying command, 3 more tries remain >>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>> (probe0:mfi0:0:0:0): Retrying command, 2 more tries remain >>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>> (probe0:mfi0:0:0:0): Retrying command, 1 more tries remain >>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>> (probe0:mfi0:0:0:0): Retrying command, 0 more tries remain >>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>> (probe0:mfi0:0:0:0): Error 5, Retries exhausted >>> >>> mfi0 Physical Drives: >>>   0 (  932G) UNCONFIGURED GOOD >> serial=ZA1AC912> SATA E1:S3 >> >> mfi(4) are RAIDs, not HBAs.  We do not recommend RAIDs with TrueNAS >> due to problems with hot-plug, disk identification, etc. and so have >> limited experience with them.  But I know some of LSI RAIDs can be >> reflashed into equivalent HBAs, so if they share the hardware, I can >> speculate that they may share some issues. > I bought 2 of these: > https://www.ebay.com/itm/194910024856 > to replace the 2 mfi(4)'s > > Hopefully I can just move the controllers and TrueNAS 13.0-RELEASE will > just notice them. > and pick up the new 8T disks. > > let me know if I'm setting myself up for failure. Not so obvious. They should work at least on a short run. But as I have told, I've heard number of rumors about mps(4) and 8TB Seagates. Though nothing confirmed, only some instability, that nobody is really motivated to debug now. Next 93xx line of mpr(4) HBAs still seems cost few times more on EBay thogh, so there is a choice. -- Alexander Motin