From owner-freebsd-fs@freebsd.org Sun May 10 00:52:14 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C44912D3B48 for ; Sun, 10 May 2020 00:52:14 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670067.outbound.protection.outlook.com [40.107.67.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49KQWd1sNQz4Wgx for ; Sun, 10 May 2020 00:52:12 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PNIL78NfQu+DXrNEcw14AV2mHAhYEBg/zEgb/yEsBNZ86SI5tUYMeVO2pSZtj8Vte6TEOqHY9oMhMZREP6WQdnpf/La6d7Nl9p/1i0HU71AeD96cfdplSsKf4LhH7tqZrwNgnUQvAKm1dratgiXQDnxs0vlxCQB6ILYVhE7ZEku43Deth5Vs3vKWHAFoUejQqWFbAspiwfnrVbHPaEKXGGmMZuYeCOl50q+9r7zKa/2UrB8LpkEQhJHTs2XpwnMv/WDfPGkiWo/cw9uUwxkT9irOM9Dmgo/l/4L6LMKLEuyR5enEpfxejVnVJfSenslqP95/uJWi4Akbb7nVg3Sy9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=adfocIdWiy2GOliL2bj9bx0olqlmtJXDmb/WRbdExug=; b=VcxHnh3ziz9Qg2bvog1walaE+ZTHAWbUMIgBJ28gK4uwBmKLP3FVcTgOp59x1WgSRvVAaigx/sGuCAZdkXKFfoKe9qKYt2yU1n21zaaZV+wjfOkIMTWCxAJ2ywue6JaoAUCieQpuLvmUhT2jcy5neHPa1v1zsYrduPC0jnpm9k0xJ0C0ZtJkZnvUIOAWdi5AtmXZLRHh0kcCXyPEjCRguuVFoSUjayydn/L/NYdeMRFMsGHXy7wLcXw/Naih+VOMiqyQ+WzlLWSeRqGKAeGsOJkVfXY9AGbyb+APIFUuIWubaG7Egt0I8qp5WZ0Hpj651vfC2caaXg/FOpQ2voG1WA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none Received: from QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM (52.132.86.26) by QB1PR01MB3137.CANPRD01.PROD.OUTLOOK.COM (52.132.84.88) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.34; Sun, 10 May 2020 00:52:11 +0000 Received: from QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM ([fe80::dd96:945c:b6ee:ffa2]) by QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM ([fe80::dd96:945c:b6ee:ffa2%6]) with mapi id 15.20.2979.028; Sun, 10 May 2020 00:52:11 +0000 From: Rick Macklem To: "freebsd-fs@FreeBSD.org" Subject: nfs-over-tls ready for testing Thread-Topic: nfs-over-tls ready for testing Thread-Index: AQHWJmQfMBxSsvId/0yJXTlHkpREXA== Date: Sun, 10 May 2020 00:52:11 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: a7872bf1-39f3-4d1b-6f55-08d7f47c5f9f x-ms-traffictypediagnostic: QB1PR01MB3137: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-forefront-prvs: 039975700A x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: gZxShJ8mK6bXcPxNKKEipTC1LxAp/9KJ62kXFLAVw148ZfzL/MM6cxa4s1ovS9ePDjsYzftQ16/RMeYw0TZ6GQ9rdTK02y3+CZqrespO6M1VQtmoOFJ5XGCzB36aII4M/5qY5UZQFOWqDnUeh1MWWAC65ZYWuZb0b/ohjw4jToD3SztK6gNFVz+ptYImlJD2hIyGkPl0SPwlLdOOh31Qo52Aqq2xoOebIlktYUiDK9pluwozaEAuYRxrByFOBJgpqb01yRMJHH4NPbZvC8BKbSOjewwtY/guJEV1I6UpzgNPMARKOfDbOaQIsZ/D8L3DLwX8BjC9YFPiu75gWDjV3+t+cE58DX0BSnMs56jD3t6aO1yS0M070eJGCVc40DGXhOd+trgxISQbH7Dy7zk4UDI1SFmQe472o/tmmGqvWMe9qJ83382QVyKHkgnJD6/lC3Rgw4FbcxrrtWf3YTG4g5iEDm7FWEcc/M+oYBchgRN4Nnth+ACdLPTXiL1rXvpHExa72vFueSKX/qePIk6QUETP40WAiixaBD2VkV1G1PJZY//GWxAzsny4i4EbIGYtVsSfZF1S4Q0ll1LriO0LABo73vjRnIFpRSwKN4mwjKs= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFTY:; SFS:(396003)(39850400004)(136003)(376002)(366004)(346002)(33430700001)(966005)(786003)(316002)(6916009)(5660300002)(9686003)(7696005)(55016002)(8936002)(86362001)(66574014)(76116006)(33440700001)(8676002)(186003)(66556008)(52536014)(66446008)(6506007)(2906002)(71200400001)(66946007)(64756008)(478600001)(33656002)(66476007); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: 2QWuxj9DICad9cxIIEbTqFhVU3+pLo6TcMB7SWCNqZxF3YLa5otdVd0cEyfDxM/qja9kZLa2OB1uZMZ4DrNX5X++akuPti6TuDi94wRpCNBmGY9ejzY4yP8OPqT/gL/9RlxQCy4jznNUuenm25sCsdPZ5vtNFbn5egTTb8Gzxtqqb4p0EjMz+XMvnnGTJU0OFg6mGGNe491YXgWrg5OYuiHGZs2F+t0sRfPuzcd07HhUGqEzPQaW9hJLkQQ4Ts85MsBnXuVDlHRnDyOGaaoYxcoKLINJONxHBmz1dn7fZ+5YoynqYD9NK1NrakR2IgOeYEejbVbwB53/UkQllFmW0oee3ROqJp0o7cvEIiQNP6AeXNTdenLzylp8+y5iSpDhbcEbhkidEEyHNxe4VO8KXtPZPn7S4xFHn58K2sNJhnrzVmUTzxtNWzzA8dlHbZV3sb14EytEeSO0hGMq4MybVjJfeahFZTcsFIKh9xKP20IbJftE+NprejCVNuxmLuWJ0j0ZjAxgJjS5xcgBlbL/BP/4vg3y51Eg/0FvABFT5rbqWzBod9h9fVHYk/J1yYjw x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: a7872bf1-39f3-4d1b-6f55-08d7f47c5f9f X-MS-Exchange-CrossTenant-originalarrivaltime: 10 May 2020 00:52:11.2321 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: iKF2bTBG+ry4McHq3lXMS8GmFvfXxb3VQ2Isf6KtIL5EWd7DXnXzL51pwX6qNs9qWUCRjgruH/zeuG7XWk2axw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: QB1PR01MB3137 X-Rspamd-Queue-Id: 49KQWd1sNQz4Wgx X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.67.67 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-4.80 / 15.00]; RCVD_TLS_LAST(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_IN_DNSWL_LOW(-0.10)[67.67.107.40.list.dnswl.org : 127.0.3.1]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[uoguelph.ca]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.40)[ipnet: 40.64.0.0/10(-3.73), asn: 8075(-3.24), country: US(-0.05)]; TO_DN_EQ_ADDR_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[67.67.107.40.rep.mailspike.net : 127.0.0.17]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:8075, ipnet:40.64.0.0/10, country:US]; MIME_TRACE(0.00)[0:+]; ARC_ALLOW(-1.00)[i=1] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.32 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 May 2020 00:52:14 -0000 Hi,=0A= =0A= I think the nfs-over-tls project is now ready for testing by others.=0A= (This uses a TLS session to encrypt/decrypt NFS RPCs on the wire.=0A= There is an internet draft called "Towards Remote Procedure=0A= Call Encryption By Default" which should soon become an RFC=0A= that describes what this implements.=0A= =0A= The biggest caveat is that the KERN_TLS does not yet support TLS1.3,=0A= so the code currently uses TLS1.2, which is not allowed by the above=0A= draft. I know jhb@ is working on TLS1.3 support, so this should get=0A= resolved.=0A= =0A= There is a basic setup document here:=0A= http://people.freebsd.org/~rmacklem/nfs-over-tls-setup.txt=0A= (It can also be found on FreeBSD's subversion repository at=0A= base/projects/nfs-over-tls.)=0A= =0A= For now, the setup takes some fiddling, but that will get easier=0A= as some of the code finds its way into head.=0A= =0A= I do hope that this can make it into FreeBSD13.=0A= =0A= Last, but not least, thanks go to jhb@ (and others, I'd guess?) for the KER= N_TLS=0A= work and for providing the ktls rx patch plus the patched openssl3=0A= needed to make it work.=0A= =0A= Let me know how it goes if you test it, rick=0A= From owner-freebsd-fs@freebsd.org Tue May 12 15:46:15 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DD87A2F2B93 for ; Tue, 12 May 2020 15:46:15 +0000 (UTC) (envelope-from darcy@druid.net) Received: from mail.vex.net (mail.vex.net [IPv6:2605:2600:1001::44]) by mx1.freebsd.org (Postfix) with ESMTP id 49M2GH0y0gz4NyB for ; Tue, 12 May 2020 15:46:14 +0000 (UTC) (envelope-from darcy@druid.net) Received: from dilbert.druid.net (mobile-166-176-122-64.mycingular.net [166.176.122.64]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: darcy) by mail.vex.net (Postfix) with ESMTPSA id 0828188FCC for ; Tue, 12 May 2020 11:46:13 -0400 (EDT) From: D'Arcy Cain Subject: RAID - hardware vs. ZFS To: freebsd-fs@freebsd.org Autocrypt: addr=darcy@druid.net; prefer-encrypt=mutual; keydata= mDMEXno+ARYJKwYBBAHaRw8BAQdAR2o17zm/M442SONWLjdQ/8Jpro8TbnUm/m1r4MHhygS0 HUQnQXJjeSBDYWluIDxkYXJjeUBkcnVpZC5uZXQ+iJYEExYIAD4WIQQZ0YBgv7psnnP4U/Hh AAd6d7lwwgUCXno+AQIbAwUJCWYBgAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRDhAAd6 d7lwwvvTAQCfxQvyB5M06arZ2cL4LDj0wVxG9ay0PTR+qrQQ9DhevAD+I7W9XlbX6qL3gfDe A0wmuPKWrdSRWEcagXI1sGSC3Ay4OAReej4BEgorBgEEAZdVAQUBAQdAbPzu0iryH2ph9hjN L+Cu1Wo85QbDLV174RHCnjmD5WMDAQgHiH4EGBYIACYWIQQZ0YBgv7psnnP4U/HhAAd6d7lw wgUCXno+AQIbDAUJCWYBgAAKCRDhAAd6d7lwwhb/AQD/8Vd7zTK1s3g4bFCApa3W4zSqUa3/ 9trMr0wfaEOngAEA8iWgHnExoVEdLo9542+dJ8K0L7ntWdfIZLgCXK/2OQ8= Message-ID: Date: Tue, 12 May 2020 10:46:12 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="7iNEsJdAnghEoLIIgGoLK5pr2tLCg5MtW" X-Rspamd-Queue-Id: 49M2GH0y0gz4NyB X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of darcy@druid.net has no SPF policy when checking 2605:2600:1001::44) smtp.mailfrom=darcy@druid.net X-Spamd-Result: default: False [-5.85 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[64.122.176.166.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[druid.net]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_SPF_NA(0.00)[]; SIGNED_PGP(-2.00)[]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~]; ASN(0.00)[asn:19842, ipnet:2605:2600::/32, country:CA]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(-2.75)[ip: (-7.83), ipnet: 2605:2600::/32(-3.91), asn: 19842(-1.91), country: CA(-0.09)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2020 15:46:15 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --7iNEsJdAnghEoLIIgGoLK5pr2tLCg5MtW Content-Type: multipart/mixed; boundary="NCwHcl2OIXHE2Y3tCHx1FvJh6KHsuToJ6" --NCwHcl2OIXHE2Y3tCHx1FvJh6KHsuToJ6 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable I have to purchase new servers soon. I am planning on getting Proliant DL360 servers. These come with hardware RAID. I was wondering what opinions people had about hardware RAID vs. using ZFS for RAID. Is one safer than the other? What about performance? What about hot swapping? All opinions welcome. --=20 D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 788 2246 (DoD#0082) (eNTP) | what's for dinner. IM: darcy@VybeNetworks.com, VoIP: sip:darcy@druid.net --NCwHcl2OIXHE2Y3tCHx1FvJh6KHsuToJ6-- --7iNEsJdAnghEoLIIgGoLK5pr2tLCg5MtW Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYIAB0WIQQZ0YBgv7psnnP4U/HhAAd6d7lwwgUCXrrExAAKCRDhAAd6d7lw wtjbAQC5XDWpjh/ce9Y6E5wi6wzXktL3P9pg0GEZjjHSjOho5QEA70M+KwdGDTpj ZrvVphgxYhk1n7gswfQCPxVL79mXgQ8= =V4oU -----END PGP SIGNATURE----- --7iNEsJdAnghEoLIIgGoLK5pr2tLCg5MtW-- From owner-freebsd-fs@freebsd.org Tue May 12 15:51:19 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1F8212F2D16 for ; Tue, 12 May 2020 15:51:19 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f181.google.com (mail-oi1-f181.google.com [209.85.167.181]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49M2N60YqFz4PMP for ; Tue, 12 May 2020 15:51:17 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f181.google.com with SMTP id o7so18650134oif.2 for ; Tue, 12 May 2020 08:51:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=dAbHg0q92kWC5CPU+u7gYFwv7H8RKqH7Rci+51SRnRw=; b=nU26WfIvKHBr0kw7yxcMzVSS9dduckeqvSEIfju2FaYt5ZiLDLL8LepiOP/O8Pw2X5 sygSA7roWXbusDuLjtleW80VO0rUgwFx9Warid2G76XdtNtDXyCUKnTAh9E31/E+XpmJ 3pPVqTCP3m8PC9Hj0jGhIvyZXhQb+Ac8hcPVMzNwKDMJ23f70AeFiOXhRlyczkghqCsj iZNXVjlNl125yNY2iqIooVgwbJFOGS1ezkeaUuRBwHq84LO46KcQE1DoXrxYGu+gd2PF u5qLtbomKVHtcYw023Jy7YYhw2Y5RXIbT06fSj9HK3taobnMlr+MtEVy1z77poeEzfCF FWuQ== X-Gm-Message-State: AGi0PuZSMowdzS6pw3yRbt4GGpuasEedpnRCIox1H7Iy6Mf+UJYOCZJw D+Ta/dnqsjYStPoLRV3zXJY9JF/vNBEAwbbkuBA= X-Google-Smtp-Source: APiQypLfYw+TBUK/VlqSl3E/o9Nz5fMNfFU61tJCAcV5EOtjY5q5JPo1vfiiaAIo1zjcXblxRku2mHbMpKtJskCG3r0= X-Received: by 2002:aca:bf09:: with SMTP id p9mr22087309oif.55.1589298674910; Tue, 12 May 2020 08:51:14 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Tue, 12 May 2020 09:51:03 -0600 Message-ID: Subject: Re: RAID - hardware vs. ZFS To: "D'Arcy Cain" Cc: freebsd-fs X-Rspamd-Queue-Id: 49M2N60YqFz4PMP X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.167.181 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-1.21 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.961,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; NEURAL_HAM_LONG(-0.98)[-0.984,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; URI_COUNT_ODD(1.00)[3]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[181.167.85.209.list.dnswl.org : 127.0.5.0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; IP_SCORE(-0.26)[ip: (-0.45), ipnet: 209.85.128.0/17(-0.39), asn: 15169(-0.42), country: US(-0.05)]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[181.167.85.209.rep.mailspike.net : 127.0.0.17]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2020 15:51:19 -0000 These days you're probably better off with ZFS software RAID. It's fast enough (easily 1GB/s on most modern hardware), rebuilds much faster for mostly empty pools (though slower for mostly full pools), and handles hot swapping well. Unlike hardware RAID, ZFS RAID is robustly protected by a Merkle hash tree of the entire file system, so undetected corruption is virtually impossible. And most importantly, the code is accessible and open source, so problems can be fixed. I haven't heard of any serious ZFS RAID bugs, but even if one pops up it is at least possible to fix it. That's not the case with HW RAID. I've heard of multiple commercial projects that failed just because of bugs in the hardware RAID that the vendor couldn't or wouldn't fix. -Alan On Tue, May 12, 2020 at 9:46 AM D'Arcy Cain wrote: > I have to purchase new servers soon. I am planning on getting Proliant > DL360 servers. These come with hardware RAID. I was wondering what > opinions people had about hardware RAID vs. using ZFS for RAID. Is one > safer than the other? What about performance? What about hot swapping? > > All opinions welcome. > > -- > D'Arcy J.M. Cain | Democracy is three wolves > http://www.druid.net/darcy/ | and a sheep voting on > +1 416 788 2246 (DoD#0082) (eNTP) | what's for dinner. > IM: darcy@VybeNetworks.com, VoIP: sip:darcy@druid.net > > From owner-freebsd-fs@freebsd.org Tue May 12 16:06:58 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BB8E62F32FB for ; Tue, 12 May 2020 16:06:58 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [IPv6:2607:f3e0:0:3::19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "pyroxene.sentex.ca", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49M2k95J8zz4QHt for ; Tue, 12 May 2020 16:06:57 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [IPv6:2607:f3e0:0:4::29] ([IPv6:2607:f3e0:0:4:0:0:0:29]) by pyroxene2a.sentex.ca (8.15.2/8.15.2) with ESMTPS id 04CG6opS003788 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Tue, 12 May 2020 12:06:50 -0400 (EDT) (envelope-from mike@sentex.net) Subject: Re: RAID - hardware vs. ZFS To: "D'Arcy Cain" , freebsd-fs@freebsd.org References: From: mike tancsa Autocrypt: addr=mike@sentex.net; keydata= mQENBFywzOMBCACoNFpwi5MeyEREiCeHtbm6pZJI/HnO+wXdCAWtZkS49weOoVyUj5BEXRZP xflV2ib2hflX4nXqhenaNiia4iaZ9ft3I1ebd7GEbGnsWCvAnob5MvDZyStDAuRxPJK1ya/s +6rOvr+eQiXYNVvfBhrCfrtR/esSkitBGxhUkBjOti8QwzD71JVF5YaOjBAs7jZUKyLGj0kW yDg4jUndudWU7G2yc9GwpHJ9aRSUN8e/mWdIogK0v+QBHfv/dsI6zVB7YuxCC9Fx8WPwfhDH VZC4kdYCQWKXrm7yb4TiVdBh5kgvlO9q3js1yYdfR1x8mjK2bH2RSv4bV3zkNmsDCIxjABEB AAG0HW1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5uZXQ+iQFUBBMBCAA+FiEEmuvCXT0aY6hs 4SbWeVOEFl5WrMgFAlywzOYCGwMFCQHhM4AFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQ eVOEFl5WrMhnPAf7Bf+ola0V9t4i8rwCMGvzkssGaxY/5zNSZO9BgSgfN0WzgmBEOy/3R4km Yn5KH94NltJYAAE5hqkFmAwK6psOqAR9cxHrRfU+gV2KO8pCDc6K/htkQcd/mclJYpCHp6Eq EVJOiAxcNaYuHZkeMdXDuvvI5Rk82VHk84BGgxIqIrhLlkguoPbXOOa+8c/Mpb1sRAGZEOuX EzKNC49+GS9gKW6ISbanyPsGEcFyP7GKMzcHBPf3cPrewZQZ6gBoNscasL6IJeAQDqzQAxbU GjO0qBSMRgnLXK7+DJlxrYdHGXqNbV6AYsmHJ6c2WWWiuRviFBqXinlgJ2FnYebZPAfWiQ== Message-ID: Date: Tue, 12 May 2020 12:06:50 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 49M2k95J8zz4QHt X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:3::19 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [-2.71 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f3e0::/32]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; HFILTER_HELO_IP_A(1.00)[pyroxene2a.sentex.ca]; DMARC_NA(0.00)[sentex.net]; HFILTER_HELO_NORES_A_OR_MX(0.30)[pyroxene2a.sentex.ca]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-1.71)[ipnet: 2607:f3e0::/32(-4.92), asn: 11647(-3.53), country: CA(-0.09)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2020 16:06:58 -0000 On 5/12/2020 11:46 AM, D'Arcy Cain wrote: > I have to purchase new servers soon. I am planning on getting Proliant > DL360 servers. These come with hardware RAID. I was wondering what > opinions people had about hardware RAID vs. using ZFS for RAID. Is one > safer than the other? What about performance? What about hot swapping? > > All opinions welcome. Not sure what the performance would be like in comparison, but the features and flexibility of ZFS are amazing.  Snapshots, replication, integrity checking are really useful features to have.  Install on both and see if performance is there on zfs vs hardware raid. If they are both good enough performance wise, go for zfs. The feature set is just too useful to pass up. If you go the zfs route, make sure your disk controller is set to JBOD. Dont install zfs on top of raid anything.     ---Mike From owner-freebsd-fs@freebsd.org Tue May 12 16:08:06 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7E8BC2F339E for ; Tue, 12 May 2020 16:08:06 +0000 (UTC) (envelope-from gta@umn.edu) Received: from mta-p6.oit.umn.edu (mta-p6.oit.umn.edu [134.84.196.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 49M2lT43Rwz4QNn for ; Tue, 12 May 2020 16:08:05 +0000 (UTC) (envelope-from gta@umn.edu) Received: from localhost (unknown [127.0.0.1]) by mta-p6.oit.umn.edu (Postfix) with ESMTP id 49M2lS2nY0z9vK1y for ; Tue, 12 May 2020 16:08:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at umn.edu Received: from mta-p6.oit.umn.edu ([127.0.0.1]) by localhost (mta-p6.oit.umn.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Hym_5NfrIWxM for ; Tue, 12 May 2020 11:08:04 -0500 (CDT) Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mta-p6.oit.umn.edu (Postfix) with ESMTPS id 49M2lS0rj7z9vCD7 for ; Tue, 12 May 2020 11:08:04 -0500 (CDT) DMARC-Filter: OpenDMARC Filter v1.3.2 mta-p6.oit.umn.edu 49M2lS0rj7z9vCD7 DKIM-Filter: OpenDKIM Filter v2.11.0 mta-p6.oit.umn.edu 49M2lS0rj7z9vCD7 Received: by mail-qt1-f199.google.com with SMTP id d8so14684299qtq.0 for ; Tue, 12 May 2020 09:08:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=551qQS0NNBXFBWZlCsss+nv6qehawMz4dRgxCmWC+e8=; b=iiDdFMGdNP6a3IRHuMHBLrEplzLTGk/galxhfwDcAkiruae8p5HnBx/axiiM6Brpa4 OK0Q42OQI9y0Ph/fnWGJisi9KwHMVveq0Hl0L93XJ9K4fUCf6QAI/jnoRlSA4mcT4e2s CwxccH0qnK5IXBUh6ApcXvqZNVcGl2F5xq+lWe9Tgm6cquzH75Kh0e8TfqIG8WQZaVk6 jXLb04H2THR2O5tBLDo+YZ+Pi2YS8WwqdYA90025ScMPizmX0bpHn/5ug9cPz9dSMaea Xsu8/fCBXgAOAnpBqR7OD85ITpC/WkOTqhm29ZY2bDJZCyzyXvgB0aNHAVb+d1dzWE9F GB5A== X-Gm-Message-State: AGi0PuYsum5PudtHhw0/MAisAuwhfTWXKNwcecpjIQoum0JWn2C1y6Pi eP1DUk3BNSj3dCny3s2Fb8olKv9OD2C4EoV14Sq+yXK9YneK8+jZSCVFnjj/GE/6gYAWjD6dtpW NYlju+11InRXZHU0rdQky8u290scUgdhjpyPs5MGQnP/v/fKhatQ1kM+nIigUWw== X-Received: by 2002:ac8:7606:: with SMTP id t6mr21975149qtq.331.1589299683172; Tue, 12 May 2020 09:08:03 -0700 (PDT) X-Google-Smtp-Source: APiQypLURIp2b/uIqMrlQAeuafck9HTJp45ka1qfo8ZEBdOB8G2jaB7RaeKjTIUP3z9v/x202EHj5A== X-Received: by 2002:ac8:7606:: with SMTP id t6mr21975109qtq.331.1589299682739; Tue, 12 May 2020 09:08:02 -0700 (PDT) Received: from ?IPv6:2601:447:c300:ee5:c485:601f:6dec:4d6? ([2601:447:c300:ee5:c485:601f:6dec:4d6]) by smtp.gmail.com with ESMTPSA id u3sm7232373qtk.63.2020.05.12.09.08.00 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 12 May 2020 09:08:01 -0700 (PDT) Subject: Re: RAID - hardware vs. ZFS To: freebsd-fs@freebsd.org References: From: Graham Allan Message-ID: <5782dcc1-3375-c693-246c-65eda7aebc29@umn.edu> Date: Tue, 12 May 2020 11:07:57 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 49M2lT43Rwz4QNn X-Spamd-Bar: ------ X-Spamd-Result: default: False [-6.46 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[umn.edu:s=google]; RCVD_COUNT_FIVE(0.00)[6]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:134.84.196.192/27]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_DN_NONE(0.00)[]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[umn.edu.dwl.dnswl.org : 127.0.11.1]; IP_SCORE(-1.76)[ipnet: 134.84.0.0/16(-4.96), asn: 217(-3.79), country: US(-0.05)]; RCVD_IN_DNSWL_MED(-0.20)[206.196.84.134.list.dnswl.org : 127.0.11.2]; DKIM_TRACE(0.00)[umn.edu:+]; DMARC_POLICY_ALLOW(-0.50)[umn.edu,quarantine]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:217, ipnet:134.84.0.0/16, country:US]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2020 16:08:06 -0000 I have implemented ZFS on HPE SmartArray controllers, but not really out of choice - it was the hardware available. I'd prefer just to use a JBOD controller. You lose all the benefits of ZFS knowledge of hardware and drive state, but of course there are useful benefits to ZFS outside of that (volume management, snapshots etc) so it's not a total loss. When I did create a ZFS pool on the hardware RAID, I just used a single large hardware RAID volume (I didn't try to expose the individual drives either as single-drive RAID-0 volumes, or drive pass-through). I found the drive pass-through on Gen9 SmartArray controllers to be very flaky, though the Gen10 ones look like they might be better. On 5/12/2020 10:46 AM, D'Arcy Cain wrote: > I have to purchase new servers soon. I am planning on getting Proliant > DL360 servers. These come with hardware RAID. I was wondering what > opinions people had about hardware RAID vs. using ZFS for RAID. Is one > safer than the other? What about performance? What about hot swapping? > > All opinions welcome. > From owner-freebsd-fs@freebsd.org Tue May 12 16:16:48 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 340532F378B for ; Tue, 12 May 2020 16:16:48 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id 49M2xV3vS4z4R0b for ; Tue, 12 May 2020 16:16:46 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Tue, 12 May 2020 18:16:12 +0200 Received-SPF: SoftFail (connect.ultra-secure.de: domain of ultra-secure.de does not designate 127.0.0.10 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=127.0.0.10; helo=connect.ultra-secure.de; envelope-from= Received: from connect.ultra-secure.de (webmail [127.0.0.10]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id 69757C22-4951-44E7-892B-44194080EEA8.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=AES256-SHA verify=NO); Tue, 12 May 2020 18:16:02 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Tue, 12 May 2020 18:16:02 +0200 From: rainer@ultra-secure.de To: Graham Allan Cc: freebsd-fs@freebsd.org Subject: Re: RAID - hardware vs. ZFS In-Reply-To: <5782dcc1-3375-c693-246c-65eda7aebc29@umn.edu> References: <5782dcc1-3375-c693-246c-65eda7aebc29@umn.edu> Message-ID: <897879c113836746e113d8a97d02d187@ultra-secure.de> X-Sender: rainer@ultra-secure.de User-Agent: Roundcube Webmail/1.2.0 X-Haraka-GeoIP: --, , NaNkm X-Haraka-GeoIP-Received: X-Haraka-p0f: os="undefined undefined" link_type="undefined" distance=undefined total_conn=undefined shared_ip=Y X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.2 required=5.0 tests=ALL_TRUSTED,BAYES_00, SPF_SOFTFAIL,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 1183, bad: 0, connections: 1189, history: 1183, pass:all_good, relaying X-Rspamd-Queue-Id: 49M2xV3vS4z4R0b X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of rainer@ultra-secure.de designates 88.198.71.201 as permitted sender) smtp.mailfrom=rainer@ultra-secure.de X-Spamd-Result: default: False [-1.78 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.89)[-0.886,0]; IP_SCORE(0.27)[ip: (-0.46), ipnet: 88.198.0.0/16(3.33), asn: 24940(-1.48), country: DE(-0.02)]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-0.97)[-0.971,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[ultra-secure.de]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FROM_NO_DN(0.00)[]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:88.198.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2020 16:16:48 -0000 Am 2020-05-12 18:07, schrieb Graham Allan via freebsd-fs: > I have implemented ZFS on HPE SmartArray controllers, but not really > out of choice - it was the hardware available. I'd prefer just to use > a JBOD controller. You lose all the benefits of ZFS knowledge of > hardware and drive state, but of course there are useful benefits to > ZFS outside of that (volume management, snapshots etc) so it's not a > total loss. > > When I did create a ZFS pool on the hardware RAID, I just used a > single large hardware RAID volume (I didn't try to expose the > individual drives either as single-drive RAID-0 volumes, or drive > pass-through). I found the drive pass-through on Gen9 SmartArray > controllers to be very flaky, though the Gen10 ones look like they > might be better. What Gen9 controllers do is "authenticate" the drive first (or try to). This means that when you first insert a replacement-drive, the controller checks if it's a genuine HPE drive. The driver (at least in 11.3) interprets this as some kind of rejection and subsequently, the drive is ignored. You have to reboot to make the drive known to FreeBSD. Gen10 use the SmartPQI driver and I think it works better. (P208/P408 HBAs). HOWEVER: make sure you download the latest firmware update DVD (https://spp.hpe.com) and apply it. These things were unstable as hell otherwise. Also, because there is no way to update the firmware from the OS, you need to boot from the ISO/USB. In a server with three SmartPQI HBAs, a SAS expander and 32 disks (partly in an external shelf), updating all firmware (which was a couple of revisions behind for HBA, NICs, ILO etc.) took over an hour for a DL380Gen10. From owner-freebsd-fs@freebsd.org Tue May 12 17:29:56 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A1A552F55C2 for ; Tue, 12 May 2020 17:29:56 +0000 (UTC) (envelope-from pen@lysator.liu.se) Received: from mail.lysator.liu.se (mail.lysator.liu.se [130.236.254.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 49M4Ys6VBdz4XG6 for ; Tue, 12 May 2020 17:29:53 +0000 (UTC) (envelope-from pen@lysator.liu.se) Received: from mail.lysator.liu.se (localhost [127.0.0.1]) by mail.lysator.liu.se (Postfix) with ESMTP id F25254001C for ; Tue, 12 May 2020 19:29:48 +0200 (CEST) Received: by mail.lysator.liu.se (Postfix, from userid 1004) id DD9FF40008; Tue, 12 May 2020 19:29:48 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on bernadotte.lysator.liu.se X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED, AWL, HTML_MESSAGE autolearn=disabled version=3.4.2 X-Spam-Score: -1.0 Received: from [IPv6:2001:9b1:28ff:d901:e4ec:5280:30d5:df52] (unknown [IPv6:2001:9b1:28ff:d901:e4ec:5280:30d5:df52]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.lysator.liu.se (Postfix) with ESMTPSA id 959EA40008; Tue, 12 May 2020 19:29:46 +0200 (CEST) From: Peter Eriksson Message-Id: <6D707E39-7213-44CE-96DA-060C3CB65843@lysator.liu.se> Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\)) Subject: Re: RAID - hardware vs. ZFS Date: Tue, 12 May 2020 19:29:46 +0200 In-Reply-To: <5782dcc1-3375-c693-246c-65eda7aebc29@umn.edu> To: freebsd-fs References: <5782dcc1-3375-c693-246c-65eda7aebc29@umn.edu> X-Mailer: Apple Mail (2.3608.80.23.2.2) X-Virus-Scanned: ClamAV using ClamSMTP X-Rspamd-Queue-Id: 49M4Ys6VBdz4XG6 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=liu.se; spf=pass (mx1.freebsd.org: domain of pen@lysator.liu.se designates 130.236.254.3 as permitted sender) smtp.mailfrom=pen@lysator.liu.se X-Spamd-Result: default: False [-4.74 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+a:mail.lysator.liu.se]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; URI_COUNT_ODD(1.00)[7]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCVD_IN_DNSWL_MED(-0.20)[3.254.236.130.list.dnswl.org : 127.0.11.2]; DMARC_POLICY_ALLOW(-0.50)[liu.se,none]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:2843, ipnet:130.236.0.0/16, country:SE]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(-3.24)[ip: (-8.47), ipnet: 130.236.0.0/16(-4.26), asn: 2843(-3.41), country: SE(-0.03)] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2020 17:29:56 -0000 We are running a number of HP DL380G9 servers with HP H241 =E2=80=9CSmart = HBA=E2=80=9D controllers in "JBOD mode=E2=80=9D and HP D6020 external = SAS disk cabinets (70 drives per cabinet (10 & 12TB drives)) using = FreeBSD 11.3-12.1 and ZFS. I would (and do) definitely stay away from RAID mode and use single disk = JBODs instead (if possible). ZFS handles errors in a much better/more = secure way than typical hardware RAID controllers since it has = =E2=80=9Cknowledge=E2=80=9D of the data that is stored - something a = RAID controller never can have. As other have stated the HP =E2=80=9CSmart=E2=80=9D controllers (using = the FreeBSD =E2=80=9Cciss=E2=80=9D device driver) are a bit = =E2=80=9Cfinicky=E2=80=9D=E2=80=A6 They (the HP H241 controller at = least) behave a bit better when put into a =E2=80=9Cpure=E2=80=9D JBOD = mode (a setting you can set from the BIOS setup menu). Before we found = that setting they were a pain to use. I _think_ you can put the modern HP =E2=80=9CSmart RAID=E2=80=9D = controllers (Pxxx) also into that JBOD mode but I haven=E2=80=99t tried = it. (I also run couple of old DL380G5 with a HP P400 controller but that = one doesn=E2=80=99t support the JBOD mode so there I have to fake it = with single-disk logical volumes). The only thing to watch out for (that I=E2=80=99ve ran into so far) with = many disks on HP controllers is a couple of bugs in the =E2=80=9Cciss=E2=80= =9D device driver which prevents it from detecting more than about 48 = drives per controller when in JBOD mode (at least with the H241 = controller). (The =E2=80=9Cciss=E2=80=9D device driver incorrectly puts a cap on = =E2=80=9Cmax_target=E2=80=9D (drive =E2=80=9Cid:s=E2=80=9D to probe to = the number of logical volumes the controller supports. For the H241 that is 64. And it seems to start to enumerate physical = disks from target 16 so 64-16 =3D a 48 drives limit=E2=80=A6 It also incorrectly sets the =E2=80=9CInitiator_id=E2=80=9D to = =E2=80=9Cmax_logical_volumes=E2=80=9D so any physical drive that happens = to get the same target number is silently skipped. I=E2=80=99ve patched the ciss driver source code and there is a bug = report so I=E2=80=99m hoping someone comes around to it and applies it = to the normal release kernel eventually (we try to stay away from custom = kernels if possible)=E2=80=A6 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D246279 = (This is not a problem unless you have many physical drives behind a = single controller in JBOD mode. We didn=E2=80=99t really notice this = until we got our second fully stocked D6020 cabinet and tried to use = both at the same time=E2=80=A6 ) We=E2=80=99=E2=80=99ve been using these servers for a couple of years = now and it works fine. That said - if I could choose I=E2=80=99d go for = an LSI HBA controller instead of the HP ones, but HP apparently = doesn=E2=80=99t sell those. (We use the Dell-branded LSI SAS3008 (Dell = HBA330) controllers in our Dell servers and they are really nice). - Peter > On 12 May 2020, at 18:07, Graham Allan via freebsd-fs = wrote: >=20 > I have implemented ZFS on HPE SmartArray controllers, but not really = out of choice - it was the hardware available. I'd prefer just to use a = JBOD controller. You lose all the benefits of ZFS knowledge of hardware = and drive state, but of course there are useful benefits to ZFS outside = of that (volume management, snapshots etc) so it's not a total loss. >=20 > When I did create a ZFS pool on the hardware RAID, I just used a = single large hardware RAID volume (I didn't try to expose the individual = drives either as single-drive RAID-0 volumes, or drive pass-through). I = found the drive pass-through on Gen9 SmartArray controllers to be very = flaky, though the Gen10 ones look like they might be better. >=20 > On 5/12/2020 10:46 AM, D'Arcy Cain wrote: >> I have to purchase new servers soon. I am planning on getting = Proliant >> DL360 servers. These come with hardware RAID. I was wondering what >> opinions people had about hardware RAID vs. using ZFS for RAID. Is = one >> safer than the other? What about performance? What about hot = swapping? >> All opinions welcome. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Wed May 13 00:28:40 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 828B92E2192 for ; Wed, 13 May 2020 00:28:40 +0000 (UTC) (envelope-from y.freebsd@paritcher.com) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 49MFs34vMrz4HX8 for ; Wed, 13 May 2020 00:28:39 +0000 (UTC) (envelope-from y.freebsd@paritcher.com) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id F17EB5C00CA; Tue, 12 May 2020 20:28:38 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Tue, 12 May 2020 20:28:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=paritcher.com; h=from:subject:to:message-id:date:mime-version:content-type; s= fm1; bh=8B6UOcKBnZPGKpNN0xi1JyqJ6Zj1Xi07Gv0cbhA5c40=; b=TjUPWp5S b/Mi6DB9tOMrTDc/AF1/XlJFeVyGGpAA+dayreY45lV16TO6TO2ydAvvYi1GVtoB yC/AKxgJyFUqsRmLhBhst2UmsmieKSPTMNhFMWiftkwfA4M4+vA9z7ZnTt5qoSdK b34C1D/5qOsqArUD+Mb193CSEN2vHu8sgKEMU1WzEGscdSw2pwcRQmGuka1epPnd DCwgfe/QIE3Qs3EV4CyqJVkmmpJNtCcXsAuwmnFR4hWbgCWC8ue+174NCgj8YiQ3 TBd0b6Jqyi5Kmb16OMbY8aN8qGaMn6ScM+0KIOp0X60EOYq06mvIbnStfkrl35sg 13BYmoRhNeUXOA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:message-id :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; bh=8B6UOcKBnZPGKpNN0xi1JyqJ6Zj1X i07Gv0cbhA5c40=; b=fCBaBUsHhMxVPUa5UR0HiXglJS6mEDgs+xWwUzsf2ryXM nN8MZXAOkUj/OqKtfHLBmd2PeTkDvniJoRMkBSk9yAHrIxmMupz4O+lTyvo/6Bm7 /Itdv+ZLyvWsusz16DnsEf3MepfVIZtn+Pcjht4621ZAhqeVdmAZCyBMZXH88gpy iHLBgRc4oIu1fGyH0noV5Cx2gWabbyPARyzIXdZ1LpA7zeaNCmhbuRA1IdZjYTYv e+iWB20NXvWPec1ZodpW9gHnbaAPVTfH2kl4o5CBp/0fl6o3x+ceRJ2rh4JEmaVd 4cgEIgH+TGN8bkm/7u2WniMX+MsDnVbIzz8K8t7TA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrleefgdefhecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffuvffkffgfgggtsegrtderredtfe ejnecuhfhrohhmpegjucfrrghrihhttghhvghruceohidrfhhrvggvsghsugesphgrrhhi thgthhgvrhdrtghomheqnecuggftrfgrthhtvghrnhephfffkeejffevgfdvkefhfedvve ehhfdtkeehffeikeehtdejhfelteejhffhtdeinecuffhomhgrihhnpehfrhgvvggsshgu rdhorhhgnecukfhppeeihedruddvkedrudelkedrudegjeenucevlhhushhtvghrufhiii gvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpeihrdhfrhgvvggsshgusehprghrihht tghhvghrrdgtohhm X-ME-Proxy: Received: from [192.168.1.201] (65-128-198-147.mpls.qwest.net [65.128.198.147]) by mail.messagingengine.com (Postfix) with ESMTPA id 750B23280078 for ; Tue, 12 May 2020 20:28:38 -0400 (EDT) From: Y Paritcher Subject: miscalculation in sa_find_sizes() To: freebsd-fs@freebsd.org Message-ID: <15b7b666-4137-a4cf-d742-dd0123ff6340@paritcher.com> Date: Tue, 12 May 2020 19:28:38 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0 MIME-Version: 1.0 Content-Language: en-US X-Rspamd-Queue-Id: 49MFs34vMrz4HX8 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=paritcher.com header.s=fm1 header.b=TjUPWp5S; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=fCBaBUsH; dmarc=pass (policy=none) header.from=paritcher.com; spf=pass (mx1.freebsd.org: domain of y.freebsd@paritcher.com designates 66.111.4.25 as permitted sender) smtp.mailfrom=y.freebsd@paritcher.com X-Spamd-Result: default: False [-5.59 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[paritcher.com:s=fm1,messagingengine.com:s=fm2]; RECEIVED_SPAMHAUS_PBL(0.00)[147.198.128.65.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.10]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[25.4.111.66.rep.mailspike.net : 127.0.0.18]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.25:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_DN_NONE(0.00)[]; URI_COUNT_ODD(1.00)[3]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[4]; IP_SCORE(-3.49)[ip: (-9.82), ipnet: 66.111.4.0/24(-4.89), asn: 11403(-2.69), country: US(-0.05)]; DKIM_TRACE(0.00)[paritcher.com:+,messagingengine.com:+]; DMARC_POLICY_ALLOW(-0.50)[paritcher.com,none]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[25.4.111.66.list.dnswl.org : 127.0.5.1] Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2020 00:28:40 -0000 Hi, I am new to FreeBSD, I reported in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=244899 there is a miscalculation in sa_find_sizes() in sa.c This issue was fixed by ZoL. How should i go about trying to get this patched in FreeBsd 11 12 & 13 ? Thanks, Yechiel From owner-freebsd-fs@freebsd.org Wed May 13 07:49:57 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B92B62EDC5C for ; Wed, 13 May 2020 07:49:57 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf1-f42.google.com (mail-lf1-f42.google.com [209.85.167.42]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49MRfD5STJz3DRV for ; Wed, 13 May 2020 07:49:56 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf1-f42.google.com with SMTP id x73so12788765lfa.2 for ; Wed, 13 May 2020 00:49:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:openpgp:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=7FFUSHzvs09JWn+mYoS+mrBL3LXfLkaSFqoawMIWTFk=; b=T2JoJgSzHGYMPiEPa4UxlT+aAhD1vftWAmaDMYme4a3XQqj4UIKYJTsgQNIBwsg8CI Z+jNjyEqBDEEzFiQrAu6syOhKqyeWgEYnMVHc0/JVaibwwrNpm2pFoEnonXsOovS6ORy DZmueFG7vwjAUW43P/ZE3Vqvc458EnaxEeJkmW7+VMmC6N4ta9lJTtEyd5ZWaDwCA82v vhyp2E9jLvSZzGmhGHgokOo10FsGLuINmqXa604nUfmK8QXGTJTl457bb99IOP4lYOI+ 6V0Vte3rvokonW43WqwqzjNNsPNHl7UQRgcwQS2PNAFyT5WMsVVEdjBhH3ZPDQeE9Yzi dROw== X-Gm-Message-State: AOAM53200RoKpRsjrf8+nzg++Mn+zyRqaYPd5OD9tgZGMyiu16OPxLOM y3ekEzqBBkzlqiOzfotMOE+9E6oyVZo= X-Google-Smtp-Source: ABdhPJweic3JbPj+Z23w63tEOqaPu4fuYj3uExdaoLQyfQP0CQBJQyBPvPGXOH/FjzsvzUam/IMxnw== X-Received: by 2002:ac2:4c14:: with SMTP id t20mr17008849lfq.24.1589356194735; Wed, 13 May 2020 00:49:54 -0700 (PDT) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id q125sm14060781ljb.34.2020.05.13.00.49.53 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 13 May 2020 00:49:54 -0700 (PDT) Subject: Re: miscalculation in sa_find_sizes() To: Y Paritcher , freebsd-fs@freebsd.org References: <15b7b666-4137-a4cf-d742-dd0123ff6340@paritcher.com> From: Andriy Gapon Openpgp: preference=signencrypt Autocrypt: addr=avg@FreeBSD.org; prefer-encrypt=mutual; keydata= mQINBFm4LIgBEADNB/3lT7f15UKeQ52xCFQx/GqHkSxEdVyLFZTmY3KyNPQGBtyvVyBfprJ7 mAeXZWfhat6cKNRAGZcL5EmewdQuUfQfBdYmKjbw3a9GFDsDNuhDA2QwFt8BmkiVMRYyvI7l N0eVzszWCUgdc3qqM6qqcgBaqsVmJluwpvwp4ZBXmch5BgDDDb1MPO8AZ2QZfIQmplkj8Y6Z AiNMknkmgaekIINSJX8IzRzKD5WwMsin70psE8dpL/iBsA2cpJGzWMObVTtCxeDKlBCNqM1i gTXta1ukdUT7JgLEFZk9ceYQQMJJtUwzWu1UHfZn0Fs29HTqawfWPSZVbulbrnu5q55R4PlQ /xURkWQUTyDpqUvb4JK371zhepXiXDwrrpnyyZABm3SFLkk2bHlheeKU6Yql4pcmSVym1AS4 dV8y0oHAfdlSCF6tpOPf2+K9nW1CFA8b/tw4oJBTtfZ1kxXOMdyZU5fiG7xb1qDgpQKgHUX8 7Rd2T1UVLVeuhYlXNw2F+a2ucY+cMoqz3LtpksUiBppJhw099gEXehcN2JbUZ2TueJdt1FdS ztnZmsHUXLxrRBtGwqnFL7GSd6snpGIKuuL305iaOGODbb9c7ne1JqBbkw1wh8ci6vvwGlzx rexzimRaBzJxlkjNfMx8WpCvYebGMydNoeEtkWldtjTNVsUAtQARAQABtB5BbmRyaXkgR2Fw b24gPGF2Z0BGcmVlQlNELm9yZz6JAlQEEwEIAD4WIQS+LEO7ngQnXA4Bjr538m7TUc1yjwUC WbgsiAIbIwUJBaOagAULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAKCRB38m7TUc1yj+JAEACV l9AK/nOWAt/9cufV2fRj0hdOqB1aCshtSrwHk/exXsDa4/FkmegxXQGY+3GWX3deIyesbVRL rYdtdK0dqJyT1SBqXK1h3/at9rxr9GQA6KWOxTjUFURsU7ok/6SIlm8uLRPNKO+yq0GDjgaO LzN+xykuBA0FlhQAXJnpZLcVfPJdWv7sSHGedL5ln8P8rxR+XnmsA5TUaaPcbhTB+mG+iKFj GghASDSfGqLWFPBlX/fpXikBDZ1gvOr8nyMY9nXhgfXpq3B6QCRYKPy58ChrZ5weeJZ29b7/ QdEO8NFNWHjSD9meiLdWQaqo9Y7uUxN3wySc/YUZxtS0bhAd8zJdNPsJYG8sXgKjeBQMVGuT eCAJFEYJqbwWvIXMfVWop4+O4xB+z2YE3jAbG/9tB/GSnQdVSj3G8MS80iLS58frnt+RSEw/ psahrfh0dh6SFHttE049xYiC+cM8J27Aaf0i9RflyITq57NuJm+AHJoU9SQUkIF0nc6lfA+o JRiyRlHZHKoRQkIg4aiKaZSWjQYRl5Txl0IZUP1dSWMX4s3XTMurC/pnja45dge/4ESOtJ9R 8XuIWg45Oq6MeIWdjKddGhRj3OohsltKgkEU3eLKYtB6qRTQypHHUawCXz88uYt5e3w4V16H lCpSTZV/EVHnNe45FVBlvK7k7HFfDDkryLkCDQRZuCyIARAAlq0slcsVboY/+IUJdcbEiJRW be9HKVz4SUchq0z9MZPX/0dcnvz/gkyYA+OuM78dNS7Mbby5dTvOqfpLJfCuhaNYOhlE0wY+ 1T6Tf1f4c/uA3U/YiadukQ3+6TJuYGAdRZD5EqYFIkreARTVWg87N9g0fT9BEqLw9lJtEGDY EWUE7L++B8o4uu3LQFEYxcrb4K/WKmgtmFcm77s0IKDrfcX4doV92QTIpLiRxcOmCC/OCYuO jB1oaaqXQzZrCutXRK0L5XN1Y1PYjIrEzHMIXmCDlLYnpFkK+itlXwlE2ZQxkfMruCWdQXye syl2fynAe8hvp7Mms9qU2r2K9EcJiR5N1t1C2/kTKNUhcRv7Yd/vwusK7BqJbhlng5ZgRx0m WxdntU/JLEntz3QBsBsWM9Y9wf2V4tLv6/DuDBta781RsCB/UrU2zNuOEkSixlUiHxw1dccI 6CVlaWkkJBxmHX22GdDFrcjvwMNIbbyfQLuBq6IOh8nvu9vuItup7qemDG3Ms6TVwA7BD3j+ 3fGprtyW8Fd/RR2bW2+LWkMrqHffAr6Y6V3h5kd2G9Q8ZWpEJk+LG6Mk3fhZhmCnHhDu6CwN MeUvxXDVO+fqc3JjFm5OxhmfVeJKrbCEUJyM8ESWLoNHLqjywdZga4Q7P12g8DUQ1mRxYg/L HgZY3zfKOqcAEQEAAYkCPAQYAQgAJhYhBL4sQ7ueBCdcDgGOvnfybtNRzXKPBQJZuCyIAhsM BQkFo5qAAAoJEHfybtNRzXKPBVwQAKfFy9P7N3OsLDMB56A4Kf+ZT+d5cIx0Yiaf4n6w7m3i ImHHHk9FIetI4Xe54a2IXh4Bq5UkAGY0667eIs+Z1Ea6I2i27Sdo7DxGwq09Qnm/Y65ADvXs 3aBvokCcm7FsM1wky395m8xUos1681oV5oxgqeRI8/76qy0hD9WR65UW+HQgZRIcIjSel9vR XDaD2HLGPTTGr7u4v00UeTMs6qvPsa2PJagogrKY8RXdFtXvweQFz78NbXhluwix2Tb9ETPk LIpDrtzV73CaE2aqBG/KrboXT2C67BgFtnk7T7Y7iKq4/XvEdDWscz2wws91BOXuMMd4c/c4 OmGW9m3RBLufFrOag1q5yUS9QbFfyqL6dftJP3Zq/xe+mr7sbWbhPVCQFrH3r26mpmy841ym dwQnNcsbIGiBASBSKksOvIDYKa2Wy8htPmWFTEOPRpFXdGQ27awcjjnB42nngyCK5ukZDHi6 w0qK5DNQQCkiweevCIC6wc3p67jl1EMFY5+z+zdTPb3h7LeVnGqW0qBQl99vVFgzLxchKcl0 R/paSFgwqXCZhAKMuUHncJuynDOP7z5LirUeFI8qsBAJi1rXpQoLJTVcW72swZ42IdPiboqx NbTMiNOiE36GqMcTPfKylCbF45JNX4nF9ElM0E+Y8gi4cizJYBRr2FBJgay0b9Cp Message-ID: <0f528106-bc79-c11c-e767-9685d13068ef@FreeBSD.org> Date: Wed, 13 May 2020 10:49:53 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Firefox/60.0 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: <15b7b666-4137-a4cf-d742-dd0123ff6340@paritcher.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 49MRfD5STJz3DRV X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of agapon@gmail.com designates 209.85.167.42 as permitted sender) smtp.mailfrom=agapon@gmail.com X-Spamd-Result: default: False [-2.24 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; DMARC_NA(0.00)[FreeBSD.org]; RCVD_COUNT_THREE(0.00)[3]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[42.167.85.209.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; IP_SCORE(-0.24)[ip: (-0.36), ipnet: 209.85.128.0/17(-0.39), asn: 15169(-0.42), country: US(-0.05)]; FORGED_SENDER(0.30)[avg@FreeBSD.org,agapon@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[42.167.85.209.rep.mailspike.net : 127.0.0.17]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[avg@FreeBSD.org,agapon@gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[96.151.72.93.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.10] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2020 07:49:57 -0000 On 13/05/2020 03:28, Y Paritcher wrote: > Hi, > > > I am new to FreeBSD, > > I reported in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=244899 there is > a miscalculation in sa_find_sizes() in sa.c > > This issue was fixed by ZoL. > > How should i go about trying to get this patched in FreeBsd 11 12 & 13 ? Could you please point out the relevant ZoL issue / PR / commit? Thanks! -- Andriy Gapon From owner-freebsd-fs@freebsd.org Wed May 13 13:07:31 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 32DA02F64EA for ; Wed, 13 May 2020 13:07:31 +0000 (UTC) (envelope-from y.freebsd@paritcher.com) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 49MZhf2qyrz431b for ; Wed, 13 May 2020 13:07:30 +0000 (UTC) (envelope-from y.freebsd@paritcher.com) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 77CB45C010A; Wed, 13 May 2020 09:07:29 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Wed, 13 May 2020 09:07:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=paritcher.com; h=subject:to:references:from:message-id:date:mime-version :in-reply-to:content-type; s=fm1; bh=oGHIlfQ2ydhwA9g+ojBD2AHXJsq r9/byJAWdPgqSuLc=; b=Bs7HsIeydXYhdfW8eQsAd3W+HcQteHAu/UjeyjNqjmo 3djQ9D/pQNopo6PrMxTSo0sPmv2feXfeTJ4UgCR64uaLTfUZ7dia1VTxlQ62mVsc g4JnGwZgVq/SF0C+UmKr0QmDZsnLIsRaLN5viWCuneL+dkw5TSHGaPzl6yAa6XZu LG/x0KiZ57cuFCIlnntUIFc7/3CCeq40acKC8mgybQ9kIoLqy0Kdnz6zLHWgfUVb FpRaFnu/n7hRQPti3iefLxltHN2etpjnRO2oz2f9lyZOr2fd63yCBKQFlmEDsPu8 7HEg9CjWnlWEQNRII80hR6R9ch/sAXku+fFdNigWBgA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=oGHIlf Q2ydhwA9g+ojBD2AHXJsqr9/byJAWdPgqSuLc=; b=BTvNoBONSoJ5cc6TagfCH5 i/Zgn7Z96ZkCvfQ73YTUp1ScFz4mapcGg2NyBquA9rqmd7pdQme2ifqjBmNjEX76 4oCIiNZbKqExuy64qFYacQgI7lrhyqHGKfF0G9oKEiZnMcOgAcwjJ9Zqcxq+6Zqc 1u0Dlg3lC9ZpPPmhB4Htvh0ft3i48Sl6UhR7xz4iJJQRv9AJpm8jGJW0hBJdrztn HaGEOhTJifB9UKCOhwa6ezV+Q+vUjCbgOHYDVQ7Bzb7fp5CBOq2VgEtBOwa1lsWS HayP0NcdZBSq2Hglr9fc3JBHGF/74W/L3WxGfXVapFU4ErTl2xBa4fDuUtOokhfw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrleeggdeitdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepuffvfhfhkffffgggjggtsegrtderre dtfeejnecuhfhrohhmpegjucfrrghrihhttghhvghruceohidrfhhrvggvsghsugesphgr rhhithgthhgvrhdrtghomheqnecuggftrfgrthhtvghrnhepveduveelveejgeevhefgje fhudfhheffuedtgfduleeivefffefhvefggeeiueegnecuffhomhgrihhnpehgihhthhhu sgdrtghomhdpfhhrvggvsghsugdrohhrghenucfkphepieehrdduvdekrdduleekrdduge ejnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhephidr fhhrvggvsghsugesphgrrhhithgthhgvrhdrtghomh X-ME-Proxy: Received: from [192.168.1.201] (65-128-198-147.mpls.qwest.net [65.128.198.147]) by mail.messagingengine.com (Postfix) with ESMTPA id 05AC130662FC for ; Wed, 13 May 2020 09:07:28 -0400 (EDT) Subject: Re: miscalculation in sa_find_sizes() To: freebsd-fs@freebsd.org References: <15b7b666-4137-a4cf-d742-dd0123ff6340@paritcher.com> <0f528106-bc79-c11c-e767-9685d13068ef@FreeBSD.org> From: Y Paritcher Message-ID: <75a38732-b797-df86-b29f-854f6018d2fe@paritcher.com> Date: Wed, 13 May 2020 08:07:28 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0 MIME-Version: 1.0 In-Reply-To: <0f528106-bc79-c11c-e767-9685d13068ef@FreeBSD.org> Content-Language: en-US X-Rspamd-Queue-Id: 49MZhf2qyrz431b X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=paritcher.com header.s=fm1 header.b=Bs7HsIey; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=BTvNoBON; dmarc=pass (policy=none) header.from=paritcher.com; spf=pass (mx1.freebsd.org: domain of y.freebsd@paritcher.com designates 66.111.4.29 as permitted sender) smtp.mailfrom=y.freebsd@paritcher.com X-Spamd-Result: default: False [-6.59 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[paritcher.com:s=fm1,messagingengine.com:s=fm2]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[29.4.111.66.rep.mailspike.net : 127.0.0.18]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.29]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[paritcher.com:+,messagingengine.com:+]; DMARC_POLICY_ALLOW(-0.50)[paritcher.com,none]; IP_SCORE(-3.49)[ip: (-9.85), ipnet: 66.111.4.0/24(-4.89), asn: 11403(-2.69), country: US(-0.05)]; RCVD_IN_DNSWL_LOW(-0.10)[29.4.111.66.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[147.198.128.65.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.10] Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2020 13:07:31 -0000 all the details are in the attached bug report in summary: caused by r294812 https://github.com/freebsd/freebsd/commit/71d7abc46e4defbcf77033c417935b944c13084a affects 10.4, 11, 12, and 13 fixed in ZoL commit https://github.com/openzfs/zfs/commit/83021b47c2870c0ba948cbcfe08f41bd7730f5fb with further fixes https://github.com/openzfs/zfs/commit/a62d1b02e372e63862cee276185f2763f641ff10 Yechiel On 5/13/20 2:49 AM, Andriy Gapon wrote: > On 13/05/2020 03:28, Y Paritcher wrote: >> Hi, >> >> >> I am new to FreeBSD, >> >> I reported in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=244899 there is >> a miscalculation in sa_find_sizes() in sa.c >> >> This issue was fixed by ZoL. >> >> How should i go about trying to get this patched in FreeBsd 11 12 & 13 ? > Could you please point out the relevant ZoL issue / PR / commit? > Thanks! > From owner-freebsd-fs@freebsd.org Fri May 15 22:00:14 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2BD972DB2FA for ; Fri, 15 May 2020 22:00:14 +0000 (UTC) (envelope-from daveb@spectralogic.com) Received: from us-smtp-delivery-147.mimecast.com (us-smtp-delivery-147.mimecast.com [216.205.24.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.mimecast.com", Issuer "DigiCert Global CA G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49P2QP0sF6z4WYM for ; Fri, 15 May 2020 22:00:12 +0000 (UTC) (envelope-from daveb@spectralogic.com) Received: from isotope.sldomain.com (192.30.190.20 [192.30.190.20]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-331-8O3TrQ4oMLGNLjJI4skRBg-1; Fri, 15 May 2020 18:00:05 -0400 X-MC-Unique: 8O3TrQ4oMLGNLjJI4skRBg-1 Received: from isotope.sldomain.com (192.168.200.20) by isotope.sldomain.com (192.168.200.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1913.5; Fri, 15 May 2020 16:00:04 -0600 Received: from isotope.sldomain.com ([fe80::5567:5b08:1164:d2ee]) by isotope.sldomain.com ([fe80::5567:5b08:1164:d2ee%6]) with mapi id 15.01.1913.005; Fri, 15 May 2020 16:00:04 -0600 From: Dave Baukus To: "freebsd-fs@freebsd.org" Subject: zpool status Thread-Topic: zpool status Thread-Index: AQHWKwQvuHWQi9XiHkODkFNHHzz8oA== Date: Fri, 15 May 2020 22:00:03 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: spectralogic.com X-Rspamd-Queue-Id: 49P2QP0sF6z4WYM X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of daveb@spectralogic.com designates 216.205.24.147 as permitted sender) smtp.mailfrom=daveb@spectralogic.com X-Spamd-Result: default: False [-2.79 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:216.205.24.0/24]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[spectralogic.com]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4]; IP_SCORE(-0.59)[ipnet: 216.205.24.0/24(-1.36), asn: 30031(-1.56), country: US(-0.05)]; MIME_BASE64_TEXT(0.10)[]; RCVD_IN_DNSWL_NONE(0.00)[147.24.205.216.list.dnswl.org : 127.0.3.0]; TO_DN_EQ_ADDR_ALL(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[147.24.205.216.rep.mailspike.net : 127.0.0.17]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:30031, ipnet:216.205.24.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[] Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2020 22:00:14 -0000 Rm9ybWVybHksIHpwb29sIHN0YXR1cyAtdiA8cG9vbD4gd291bGQgcGxhY2UgYSAicmVzaWx2ZXJp bmciIHRhZyBpbiB0aGUgc3RhdHVzLg0KRm9yIGV4YW1wbGU6DQoNCiMgenBvb2wgc3RhdHVzIC12 IFRlc3QNCiAgcG9vbDogVGVzdA0KIHN0YXRlOiBERUdSQURFRA0Kc3RhdHVzOiBPbmUgb3IgbW9y ZSBkZXZpY2VzIGlzIGN1cnJlbnRseSBiZWluZyByZXNpbHZlcmVkLiAgVGhlIHBvb2wgd2lsbA0K ICAgICAgICBjb250aW51ZSB0byBmdW5jdGlvbiwgcG9zc2libHkgaW4gYSBkZWdyYWRlZCBzdGF0 ZS4NCmFjdGlvbjogV2FpdCBmb3IgdGhlIHJlc2lsdmVyIHRvIGNvbXBsZXRlLg0KICBzY2FuOiBy ZXNpbHZlciBpbiBwcm9ncmVzcyBzaW5jZSBUdWUgTWF5IDEyIDE2OjI5OjU2IDIwMjANCiAgICAg ICAgOTIuOEcgc2Nhbm5lZCBvdXQgb2YgMjk1RyBhdCAyOTJNL3MsIDBoMTFtIHRvIGdvDQogICAg ICAgIDQ2LjFHIHJlc2lsdmVyZWQsIDMxLjUwJSBkb25lDQpjb25maWc6DQogICAgICAgIE5BTUUg ICAgICAgICAgICAgICAgICAgICAgICBTVEFURSAgICAgUkVBRCBXUklURSBDS1NVTQ0KICAgICAg ICBUZXN0ICAgICAgICAgICAgICAgICAgICAgICAgREVHUkFERUQgICAgIDAgICAgIDAgICAgIDAN CiAgICAgICAgICBtaXJyb3ItMCAgICAgICAgICAgICAgICAgIE9OTElORSAgICAgICAwICAgICAw ICAgICAwDQogICAgICAgICAgICBkYTUgICAgICAgICAgICAgICAgICAgICBPTkxJTkUgICAgICAg MCAgICAgMCAgICAgMA0KICAgICAgICAgICAgZGEzICAgICAgICAgICAgICAgICAgICAgT05MSU5F ICAgICAgIDAgICAgIDAgICAgIDANCiAgICAgICAgICBtaXJyb3ItMSAgICAgICAgICAgICAgICAg IERFR1JBREVEICAgICAwICAgICAwICAgICAwDQogICAgICAgICAgICBkYTYgICAgICAgICAgICAg ICAgICAgICBPTkxJTkUgICAgICAgMCAgICAgMCAgICAgMA0KICAgICAgICAgICAgc3BhcmUtMSAg ICAgICAgICAgICAgICAgUkVNT1ZFRCAgICAgIDAgICAgIDAgICAgIDANCiAgICAgICAgICAgICAg MTI3NTcwMzM1NTY3MzQ4MzY5NTYgIFJFTU9WRUQgICAgICAwICAgICAwICAgICAwICB3YXMgL2Rl di9kYTgNCiAgICAgICAgICAgICAgZGE5ICAgICAgICAgICAgICAgICAgIE9OTElORSAgICAgICAw ICAgICAwICAgICAwICAocmVzaWx2ZXJpbmcpDQogICAgICAgIHNwYXJlcw0KICAgICAgICAgIDQ2 MTc3NzM4NjE5NTk2NTYyODEgICAgICAgSU5VU0UgICAgIHdhcyAvZGV2L2RhOQ0KZXJyb3JzOiBO byBrbm93biBkYXRhIGVycm9ycw0KDQpUaGUgY29kZSBpbiB6cG9vbCBpczoNCnByaW50X3N0YXR1 c19jb25maWcoKSB7DQouLi4NCi4uLg0KLi4uDQogICAgICAgICh2b2lkKSBudmxpc3RfbG9va3Vw X3VpbnQ2NF9hcnJheShudiwgWlBPT0xfQ09ORklHX1NDQU5fU1RBVFMsDQogICAgICAgICAgICAo dWludDY0X3QgKiopJnBzLCAmYyk7DQoNCiAgICAgICAgaWYgKHBzICE9IE5VTEwgJiYgcHMtPnBz c19zdGF0ZSA9PSBEU1NfU0NBTk5JTkcgJiYNCiAgICAgICAgICAgIHZzLT52c19zY2FuX3Byb2Nl c3NlZCAhPSAwICYmIGNoaWxkcmVuID09IDApIHsNCiAgICAgICAgICAgICAgICAodm9pZCkgcHJp bnRmKGdldHRleHQoIiAgKCVzKSIpLA0KICAgICAgICAgICAgICAgICAgICAocHMtPnBzc19mdW5j ID09IFBPT0xfU0NBTl9SRVNJTFZFUikgPw0KICAgICAgICAgICAgICAgICAgICAicmVzaWx2ZXJp bmciIDogInJlcGFpcmluZyIpOw0KICAgICAgICB9DQoNCkkgYmVsaWV2ZSB0aGUgYWJvdmUgY29k ZSBpcyBub3cgZGVhZCBiZWNhdXNlIHRoaXMgY2hhbmdlIG1hZGUgaXQgYSBuby1vcDoNCiAgICAg ICAgTUZWIHIzMzE2OTUsIDMzMTcwMDogOTE2NiB6ZnMgc3RvcmFnZSBwb29sIGNoZWNrcG9pbnQN CiAgICAgICAgaWxsdW1vcy9pbGx1bW9zLWdhdGVAODY3MTQwMDEzNGExMWM4NDgyNDQ4OTZjYTUx YTdkYjRkMGY2OWRhNA0KDQpUaGUgYWJvdmUgY2hhbmdlIGludHJvZHVjZWQgdGhpcyBmdW5jdGlv bjoNCnJvb3RfdmRldl9hY3Rpb25zX2dldHByb2dyZXNzKHZkZXZfdCAqdmQsIG52bGlzdF90ICpu dmwpDQp7DQogICAgICAgIHNwYV90ICpzcGEgPSB2ZC0+dmRldl9zcGE7DQogICAgICAgIGlmICh2 ZCAhPSBzcGEtPnNwYV9yb290X3ZkZXYpDQogICAgICAgICAgICAgICAgcmV0dXJuOw0KICAgICAg ICAvKiBwcm92aWRlIGVpdGhlciBjdXJyZW50IG9yIHByZXZpb3VzIHNjYW4gaW5mb3JtYXRpb24g Ki8NCiAgICAgICAgcG9vbF9zY2FuX3N0YXRfdCBwczsNCiAgICAgICAgaWYgKHNwYV9zY2FuX2dl dF9zdGF0cyhzcGEsICZwcykgPT0gMCkgew0KICAgICAgICAgICAgICAgIGZudmxpc3RfYWRkX3Vp bnQ2NF9hcnJheShudmwsDQogICAgICAgICAgICAgICAgICAgIFpQT09MX0NPTkZJR19TQ0FOX1NU QVRTLCAodWludDY0X3QgKikmcHMsDQogICAgICAgICAgICAgICAgICAgIHNpemVvZiAocG9vbF9z Y2FuX3N0YXRfdCkgLyBzaXplb2YgKHVpbnQ2NF90KSk7DQogICAgICAgIH0NCi4uLg0KfQ0KDQpB cyBvbmUgY2FuIHNlZSwgd2Ugbm93IG9ubHkgZ2V0IFpQT09MX0NPTkZJR19TQ0FOX1NUQVRTIGZv ciB0aGUgcm9vdCB2ZGV2LCB0aHVzIHRoZSBudmxpc3QgZm9yIGEgY2hpbGQgcmVzaWx2ZXJpbmcg dmRldiBuZXZlciBnZXRzIHRhZ2dlZCBieSB6cG9vbC4gQmVmb3JlIHJvb3RfdmRldl9hY3Rpb25z X2dldHByb2dyZXNzKCkgd2FzIGludHJvZHVjZWQsIHRoZSBjb2RlIGFsd2F5cyByZXRyaWV2ZWQg WlBPT0xfQ09ORklHX1NDQU5fU1RBVFMNCg0KTXkgcXVpY2sgZml4IHdvdWxkIGJlIHRvIG1vdmUg dGhlIHJvb3QgdmRldiBjaGVjayB0byBhZnRlciBzcGFfc2Nhbl9nZXRfc3RhdHMoKS4NCk9yIHJl bW92ZSB0aGUgZGVhZCBjb2RlIGluIHpwb29sX21haW4uYw0KDQotLQ0KRGF2ZSBCYXVrdXMNCg0K