From owner-freebsd-stable@freebsd.org Wed Oct 21 12:21:08 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1DF4B44B113; Wed, 21 Oct 2020 12:21:08 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CGV2q38dYz4Gbx; Wed, 21 Oct 2020 12:21:07 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: by mail-ej1-x62a.google.com with SMTP id qp15so2928659ejb.3; Wed, 21 Oct 2020 05:21:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=Q5b2iMlm72HwEw/GDSlS0dg2n0L9yw6Aumoal2DbVZw=; b=KxdsdSOYlR8FOG+JajT3gUzNaSfbxpQeqgJ6fHOLlqrbkG1PIrX5JXfwRWGNK6LxmW xuqcgdPVdHfvbel5OQCGwNJDqAvgqGCz8nPWx4kbB8l20xBkCwmKvKOJu5fmZM+yJFhy qJeDxRiYLhpav/JnlG4QMFIxqkNkSLyxzPsmdDesgjFuUOBkUVujM5Lg+0xWTnpkYia8 65rqcVKiY1WE0wr1YWP1zUduwltJgOwjJOa/u0Jsg8D9cg3DPCJeRBdb4Hjm6OxHXBnn ufml6U9JOQ9asqx67tpUmtv/6ZdjYDwtJ5VznWe0FYw+agLzGGWfq26TgaX1eNbp5ce5 ck0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=Q5b2iMlm72HwEw/GDSlS0dg2n0L9yw6Aumoal2DbVZw=; b=GpSSHIyjF1eqFi+AcChyj/6GEJJeDiR6/aVbkdDpqsJJIEvqi8RF1ro+Wl1cWQajza 5tzEQmYn9yx+CUhcYQwEyB7iKv2KpyMH/JxTlxoStfwkyTN9YgNIdpmziwRWAm0Comgp EGcCNSLAoCLiV+9L3MNV06T/tSuR/YoOCqCsUGjrVzFntVwAc3zYYEuxzOAycP2hYBKj XKx4gw2r0DKeSDMl5JGdDJz63TSI0U17oM8VTPCnEZIqC8nyL0MY2FWDNPB8V6uqL2xe 37fXGICIiionnBEh81CSwpHlzMeVHetdXe7dK+qpDsjxE4qhq7VoAWf9U/7ErZ6qLi7L OMUQ== X-Gm-Message-State: AOAM530heADzFb1+fF6NszT6Lg7fdEI1fUeLQKIff4M8WDcfmvGT1PpL HmyTBpkzIwRAAjf+22Dj5+1EC9WfmrkgwTJUk8bZqqxQ X-Google-Smtp-Source: ABdhPJxKyKNv6uJpipgiR8so3SNrRCV0zO+yfEbVCYK1pv2fFSjeSeQXAnZy996GBgkZAuC4P4rVs8fbkvzqa6TPhtk= X-Received: by 2002:a17:906:9483:: with SMTP id t3mr3399488ejx.390.1603282864298; Wed, 21 Oct 2020 05:21:04 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Cassiano Peixoto Date: Wed, 21 Oct 2020 09:20:53 -0300 Message-ID: Subject: Re: Zpool doesn't boot anymore after FreeBSD 12.1 To: freebsd-stable@freebsd.org, freebsd-current@freebsd.org X-Rspamd-Queue-Id: 4CGV2q38dYz4Gbx X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=KxdsdSOY; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of peixotocassiano@gmail.com designates 2a00:1450:4864:20::62a as permitted sender) smtp.mailfrom=peixotocassiano@gmail.com X-Spamd-Result: default: False [-3.92 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.02)[-1.022]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-0.99)[-0.990]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62a:from]; NEURAL_HAM_SHORT(-0.91)[-0.910]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable,freebsd-current]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2020 12:21:08 -0000 Hi there, Anyone can help please? I've many servers with this same issue. Thanks On Fri, Oct 16, 2020 at 10:24 AM Cassiano Peixoto wrote: > Hi there, > > I have a FreeBSD 12.1-STABLE running on VMWARE with one disk. Then I adde= d two more disks to expand my pool. BTW I already did it many time with no = issues. > > I ran: > > # zpool status > pool: zroot > state: ONLINE > status: Some supported features are not enabled on the pool. The pool can > still be used, but some features are unavailable. > action: Enable all features using 'zpool upgrade'. Once this is done, > the pool may no longer be accessible by software that does not support > the features. See zpool-features(7) for details. > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > zroot ONLINE 0 0 0 > gpt/disk0 ONLINE 0 0 0 > > errors: No known data errors > > # zpool add -f zroot da1 > # zpool add -f zroot da2 > # zpool status > pool: zroot > state: ONLINE > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > zroot ONLINE 0 0 0 > gpt/disk0 ONLINE 0 0 0 > da1 ONLINE 0 0 0 > da2 ONLINE 0 0 0 > > errors: No known data errors > # reboot > > Then my system doesn=E2=80=99t boot anymore, i got the following error: > > gptzfsboot: error 4 lba 2038346899 > gptzfsboot: error 4 lba 1361327267 > /boot/config: -Dh > > BTX loader 1.00 BTX version is 1.02 > Consoles: internal video/keyboard serial port > BIOS drive A: is fd0 > BIOS drive C: is disk0 > BIOS drive D: is disk1 > BIOS drive E: is disk2 > BIOS drive F: is disk3 > BIOS drive G: is disk4 > BIOS drive H: is disk5 > ZFS: i/o error - all block copies unavailable > ZFS: failed to read pool zroot directory object > BIOS 638kB/3143616kB available memory > > FreeBSD/x86 bootstrap loader, Revision 1.1 > ERROR: cannot open /boot/lua/loader.lua: invalid argument. > > Type '?' for list of commands, 'help' for more datailed help. > OK > > I can import my pool with no problems using the lived, but I could not fi= x it. > > Seems a bug after 12.1-STABLE. Please, anyone can take a look ok that? > > Thanks. > > > > From owner-freebsd-stable@freebsd.org Wed Oct 21 13:53:21 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D121A44CE92 for ; Wed, 21 Oct 2020 13:53:21 +0000 (UTC) (envelope-from nikolay.kostirya@i11.co) Received: from mx.i11.co (mx.i11.co [159.69.78.69]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CGX5D3Vx9z4LxN for ; Wed, 21 Oct 2020 13:53:20 +0000 (UTC) (envelope-from nikolay.kostirya@i11.co) Received: from [91.206.111.188] (helo=localhost) by mx.i11.co with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kVEYC-0005La-U0 for freebsd-stable@freebsd.org; Wed, 21 Oct 2020 13:53:13 +0000 Date: Wed, 21 Oct 2020 16:53:11 +0300 From: Nick Kostirya To: freebsd-stable@freebsd.org Subject: mmap and MAP_STACK Message-ID: <20201021165311.186bd606@i11.co> X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; i386-portbld-freebsd12.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4CGX5D3Vx9z4LxN X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.33 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[i11.co:s=omicron]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:159.69.78.69]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.008]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_MEDIUM(-1.04)[-1.044]; DKIM_TRACE(0.00)[i11.co:+]; DMARC_POLICY_ALLOW(-0.50)[i11.co,reject]; NEURAL_HAM_SHORT(-0.28)[-0.275]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:159.69.0.0/16, country:DE]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2020 13:53:21 -0000 Hello. I have question about mmap. void *OSMem::AllocateDataArea(size_t &space) { // Round up to an integral number of pages. space = (space + pageSize-1) & ~(pageSize-1); int fd = -1; // This value is required by FreeBSD. Linux doesn't care int flags = MAP_PRIVATE | MAP_ANON; #ifdef MAP_STACK if (memUsage == UsageStack) flags |= MAP_STACK; // OpenBSD seems to require this #endif void *result = mmap(0, space, PROT_READ|PROT_WRITE, flags, fd, 0); // Convert MAP_FAILED (-1) into NULL if (result == MAP_FAILED) return 0; return result; } When MAP_STACK is used, "insufficient memory" error occurs. When MAP_STACK removed, it is all right. Please tell me why. From owner-freebsd-stable@freebsd.org Wed Oct 21 14:17:12 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3F0F344D28F for ; Wed, 21 Oct 2020 14:17:12 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CGXcl2WMZz4MpH for ; Wed, 21 Oct 2020 14:17:11 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 09LEGvsY070320 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 21 Oct 2020 17:17:00 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 09LEGvsY070320 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 09LEGvug070319; Wed, 21 Oct 2020 17:16:57 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 21 Oct 2020 17:16:57 +0300 From: Konstantin Belousov To: Nick Kostirya Cc: freebsd-stable@freebsd.org Subject: Re: mmap and MAP_STACK Message-ID: <20201021141657.GJ2643@kib.kiev.ua> References: <20201021165311.186bd606@i11.co> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20201021165311.186bd606@i11.co> X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4CGXcl2WMZz4MpH X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-0.17 / 15.00]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; NEURAL_HAM_LONG(-0.12)[-0.124]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; NEURAL_HAM_MEDIUM(-0.61)[-0.606]; R_SPF_SOFTFAIL(0.00)[~all:c]; NEURAL_SPAM_SHORT(0.56)[0.559]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2020 14:17:12 -0000 On Wed, Oct 21, 2020 at 04:53:11PM +0300, Nick Kostirya via freebsd-stable wrote: > Hello. > I have question about mmap. > > void *OSMem::AllocateDataArea(size_t &space) > { > // Round up to an integral number of pages. > space = (space + pageSize-1) & ~(pageSize-1); > int fd = -1; // This value is required by FreeBSD. Linux doesn't care > int flags = MAP_PRIVATE | MAP_ANON; > #ifdef MAP_STACK > if (memUsage == UsageStack) flags |= MAP_STACK; // OpenBSD seems to require this > #endif > void *result = mmap(0, space, PROT_READ|PROT_WRITE, flags, fd, 0); > // Convert MAP_FAILED (-1) into NULL > if (result == MAP_FAILED) > return 0; > return result; > } > > > When MAP_STACK is used, "insufficient memory" error occurs. > When MAP_STACK removed, it is all right. > > Please tell me why. Show ktrace/kdump output of the mmap(2) without and with MAP_STACK. Or provide a minimal self-contained C source that demonstrates your issue. From owner-freebsd-stable@freebsd.org Wed Oct 21 15:18:56 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3055E44E4DD for ; Wed, 21 Oct 2020 15:18:56 +0000 (UTC) (envelope-from nikolay.kostirya@i11.co) Received: from mx.i11.co (mx.i11.co [159.69.78.69]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CGYzy5Hz9z4Qdc for ; Wed, 21 Oct 2020 15:18:54 +0000 (UTC) (envelope-from nikolay.kostirya@i11.co) Received: from [91.206.111.188] (helo=localhost) by mx.i11.co with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kVFt6-00060F-O5 for freebsd-stable@freebsd.org; Wed, 21 Oct 2020 15:18:52 +0000 Date: Wed, 21 Oct 2020 18:18:50 +0300 From: Nick Kostirya To: freebsd-stable@freebsd.org Subject: Re: mmap and MAP_STACK Message-ID: <20201021181850.49126cdf@i11.co> In-Reply-To: <20201021141657.GJ2643@kib.kiev.ua> References: <20201021165311.186bd606@i11.co> <20201021141657.GJ2643@kib.kiev.ua> X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; i386-portbld-freebsd12.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4CGYzy5Hz9z4Qdc X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.61 / 15.00]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[i11.co:s=omicron]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:159.69.78.69]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-0.99)[-0.994]; DKIM_TRACE(0.00)[i11.co:+]; DMARC_POLICY_ALLOW(-0.50)[i11.co,reject]; NEURAL_HAM_SHORT(-0.62)[-0.617]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:159.69.0.0/16, country:DE]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2020 15:18:56 -0000 On Wed, 21 Oct 2020 17:16:57 +0300 Konstantin Belousov wrote: > On Wed, Oct 21, 2020 at 04:53:11PM +0300, Nick Kostirya via freebsd-stable wrote: > > Hello. > > I have question about mmap. > > > > void *OSMem::AllocateDataArea(size_t &space) > > { > > // Round up to an integral number of pages. > > space = (space + pageSize-1) & ~(pageSize-1); > > int fd = -1; // This value is required by FreeBSD. Linux doesn't care > > int flags = MAP_PRIVATE | MAP_ANON; > > #ifdef MAP_STACK > > if (memUsage == UsageStack) flags |= MAP_STACK; // OpenBSD seems to require this > > #endif > > void *result = mmap(0, space, PROT_READ|PROT_WRITE, flags, fd, 0); > > // Convert MAP_FAILED (-1) into NULL > > if (result == MAP_FAILED) > > return 0; > > return result; > > } > > > > > > When MAP_STACK is used, "insufficient memory" error occurs. > > When MAP_STACK removed, it is all right. > > > > Please tell me why. > Show ktrace/kdump output of the mmap(2) without and with MAP_STACK. > > Or provide a minimal self-contained C source that demonstrates your > issue. kdump with MAP_STACK. 87183 polyimport CALL mmap(0,0x1000,0x3,0x1402,0xffffffff,0,0) 87183 polyimport RET mmap -1 errno 22 Invalid argument kdump without MAP_STACK. 93712 polyimport CALL mmap(0,0x1000,0x3,0x1002,0xffffffff,0,0) 93712 polyimport RET mmap 547053568/0x209b6000 From owner-freebsd-stable@freebsd.org Wed Oct 21 15:28:54 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5053F44EB05 for ; Wed, 21 Oct 2020 15:28:54 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CGZCT2T55z4R5D for ; Wed, 21 Oct 2020 15:28:52 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 09LFSid2086991 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 21 Oct 2020 18:28:47 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 09LFSid2086991 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 09LFSibJ086990; Wed, 21 Oct 2020 18:28:44 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 21 Oct 2020 18:28:44 +0300 From: Konstantin Belousov To: Nick Kostirya Cc: freebsd-stable@freebsd.org Subject: Re: mmap and MAP_STACK Message-ID: <20201021152844.GK2643@kib.kiev.ua> References: <20201021165311.186bd606@i11.co> <20201021141657.GJ2643@kib.kiev.ua> <20201021181850.49126cdf@i11.co> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20201021181850.49126cdf@i11.co> X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4CGZCT2T55z4R5D X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [0.21 / 15.00]; RCVD_TLS_ALL(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; NEURAL_HAM_LONG(-0.13)[-0.129]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; NEURAL_SPAM_MEDIUM(0.09)[0.089]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.25)[0.254]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2020 15:28:54 -0000 On Wed, Oct 21, 2020 at 06:18:50PM +0300, Nick Kostirya via freebsd-stable wrote: > On Wed, 21 Oct 2020 17:16:57 +0300 > Konstantin Belousov wrote: > > > On Wed, Oct 21, 2020 at 04:53:11PM +0300, Nick Kostirya via freebsd-stable wrote: > > > Hello. > > > I have question about mmap. > > > > > > void *OSMem::AllocateDataArea(size_t &space) > > > { > > > // Round up to an integral number of pages. > > > space = (space + pageSize-1) & ~(pageSize-1); > > > int fd = -1; // This value is required by FreeBSD. Linux doesn't care > > > int flags = MAP_PRIVATE | MAP_ANON; > > > #ifdef MAP_STACK > > > if (memUsage == UsageStack) flags |= MAP_STACK; // OpenBSD seems to require this > > > #endif > > > void *result = mmap(0, space, PROT_READ|PROT_WRITE, flags, fd, 0); > > > // Convert MAP_FAILED (-1) into NULL > > > if (result == MAP_FAILED) > > > return 0; > > > return result; > > > } > > > > > > > > > When MAP_STACK is used, "insufficient memory" error occurs. > > > When MAP_STACK removed, it is all right. > > > > > > Please tell me why. > > Show ktrace/kdump output of the mmap(2) without and with MAP_STACK. > > > > Or provide a minimal self-contained C source that demonstrates your > > issue. > > kdump with MAP_STACK. > > 87183 polyimport CALL mmap(0,0x1000,0x3,0x1402,0xffffffff,0,0) > 87183 polyimport RET mmap -1 errno 22 Invalid argument So it is anything but 'insufficient memory' (I suspected ENOMEM). EINVAL there is because sysctl security.bsd.stack_guard_page default value is 1, which means that at least one page of the stack is reserved as guard. Kernel does not allow to map stack that would have no data pages (all pages are guard). Your mapping request is for one page, and one page is due to guard, so you get EINVAL. Generally MAP_STACK is magic and requires caller to know what it does. > > > kdump without MAP_STACK. > > 93712 polyimport CALL mmap(0,0x1000,0x3,0x1002,0xffffffff,0,0) > 93712 polyimport RET mmap 547053568/0x209b6000 > > > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Wed Oct 21 15:47:34 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4CF3E44EF21 for ; Wed, 21 Oct 2020 15:47:34 +0000 (UTC) (envelope-from nikolay.kostirya@i11.co) Received: from mx.i11.co (mx.i11.co [159.69.78.69]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CGZd12j4mz4SC3 for ; Wed, 21 Oct 2020 15:47:33 +0000 (UTC) (envelope-from nikolay.kostirya@i11.co) Received: from [91.206.111.188] (helo=localhost) by mx.i11.co with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kVGKp-0006BW-R6 for freebsd-stable@freebsd.org; Wed, 21 Oct 2020 15:47:31 +0000 Date: Wed, 21 Oct 2020 18:47:30 +0300 From: Nick Kostirya To: freebsd-stable@freebsd.org Subject: Re: mmap and MAP_STACK Message-ID: <20201021184730.080cecb3@i11.co> In-Reply-To: <20201021152844.GK2643@kib.kiev.ua> References: <20201021165311.186bd606@i11.co> <20201021141657.GJ2643@kib.kiev.ua> <20201021181850.49126cdf@i11.co> <20201021152844.GK2643@kib.kiev.ua> X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; i386-portbld-freebsd12.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4CGZd12j4mz4SC3 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.16 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[i11.co:s=omicron]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:159.69.78.69:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_MEDIUM(-0.99)[-0.994]; DKIM_TRACE(0.00)[i11.co:+]; MID_RHS_MATCH_FROM(0.00)[]; DMARC_POLICY_ALLOW(-0.50)[i11.co,reject]; NEURAL_HAM_SHORT(-0.16)[-0.162]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:159.69.0.0/16, country:DE]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2020 15:47:34 -0000 On Wed, 21 Oct 2020 18:28:44 +0300 Konstantin Belousov wrote: > > > > kdump with MAP_STACK. > > > > 87183 polyimport CALL mmap(0,0x1000,0x3,0x1402,0xffffffff,0,0) > > 87183 polyimport RET mmap -1 errno 22 Invalid argument > So it is anything but 'insufficient memory' (I suspected ENOMEM). > EINVAL there is because sysctl security.bsd.stack_guard_page default value > is 1, which means that at least one page of the stack is reserved as guard. > Kernel does not allow to map stack that would have no data pages (all pages > are guard). > > Your mapping request is for one page, and one page is due to guard, so > you get EINVAL. Generally MAP_STACK is magic and requires caller to know > what it does. Thanks! From owner-freebsd-stable@freebsd.org Wed Oct 21 19:45:02 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4EC6E42BA5D for ; Wed, 21 Oct 2020 19:45:02 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [IPv6:2607:f3e0:0:3::19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "pyroxene.sentex.ca", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CGgv13T30z3T2X for ; Wed, 21 Oct 2020 19:45:01 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [IPv6:2607:f3e0:0:4:893:f92f:3afb:faba] ([IPv6:2607:f3e0:0:4:893:f92f:3afb:faba]) by pyroxene2a.sentex.ca (8.15.2/8.15.2) with ESMTPS id 09LJj0F2025497 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Wed, 21 Oct 2020 15:45:00 -0400 (EDT) (envelope-from mike@sentex.net) Subject: Re: Zpool doesn't boot anymore after FreeBSD 12.1 To: Cassiano Peixoto , freebsd-stable@freebsd.org References: From: mike tancsa Autocrypt: addr=mike@sentex.net; keydata= mQENBFywzOMBCACoNFpwi5MeyEREiCeHtbm6pZJI/HnO+wXdCAWtZkS49weOoVyUj5BEXRZP xflV2ib2hflX4nXqhenaNiia4iaZ9ft3I1ebd7GEbGnsWCvAnob5MvDZyStDAuRxPJK1ya/s +6rOvr+eQiXYNVvfBhrCfrtR/esSkitBGxhUkBjOti8QwzD71JVF5YaOjBAs7jZUKyLGj0kW yDg4jUndudWU7G2yc9GwpHJ9aRSUN8e/mWdIogK0v+QBHfv/dsI6zVB7YuxCC9Fx8WPwfhDH VZC4kdYCQWKXrm7yb4TiVdBh5kgvlO9q3js1yYdfR1x8mjK2bH2RSv4bV3zkNmsDCIxjABEB AAG0HW1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5uZXQ+iQFUBBMBCAA+FiEEmuvCXT0aY6hs 4SbWeVOEFl5WrMgFAlywzOYCGwMFCQHhM4AFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQ eVOEFl5WrMhnPAf7Bf+ola0V9t4i8rwCMGvzkssGaxY/5zNSZO9BgSgfN0WzgmBEOy/3R4km Yn5KH94NltJYAAE5hqkFmAwK6psOqAR9cxHrRfU+gV2KO8pCDc6K/htkQcd/mclJYpCHp6Eq EVJOiAxcNaYuHZkeMdXDuvvI5Rk82VHk84BGgxIqIrhLlkguoPbXOOa+8c/Mpb1sRAGZEOuX EzKNC49+GS9gKW6ISbanyPsGEcFyP7GKMzcHBPf3cPrewZQZ6gBoNscasL6IJeAQDqzQAxbU GjO0qBSMRgnLXK7+DJlxrYdHGXqNbV6AYsmHJ6c2WWWiuRviFBqXinlgJ2FnYebZPAfWiQ== Message-ID: Date: Wed, 21 Oct 2020 15:45:01 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4CGgv13T30z3T2X X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:3::19 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [-1.23 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[mike]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f3e0::/32]; MID_RHS_MATCH_FROM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sentex.net]; HFILTER_HELO_IP_A(1.00)[pyroxene2a.sentex.ca]; HFILTER_HELO_NORES_A_OR_MX(0.30)[pyroxene2a.sentex.ca]; NEURAL_HAM_LONG(-0.97)[-0.967]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.30)[-0.298]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-0.97)[-0.965]; FREEMAIL_TO(0.00)[gmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2020 19:45:02 -0000 Just a guess, Is your VM still trying to boot from whatever gpt/disk0 is ? Or is it perhaps trying to boot from da1 or da2 which does not have boot info ?  Generally its not recommended to use the entire disk as part of a pool. Create a partition scheme first gpart create -s gpt da1 gpart add -t freebsd-zfs da1 then add da1p1 to the pool     ---Mike On 10/21/2020 8:20 AM, Cassiano Peixoto wrote: > Hi there, > > Anyone can help please? I've many servers with this same issue. Thanks > > On Fri, Oct 16, 2020 at 10:24 AM Cassiano Peixoto > wrote: > >> Hi there, >> >> I have a FreeBSD 12.1-STABLE running on VMWARE with one disk. Then I added two more disks to expand my pool. BTW I already did it many time with no issues. >> >> I ran: >> >> # zpool status >> pool: zroot >> state: ONLINE >> status: Some supported features are not enabled on the pool. The pool can >> still be used, but some features are unavailable. >> action: Enable all features using 'zpool upgrade'. Once this is done, >> the pool may no longer be accessible by software that does not support >> the features. See zpool-features(7) for details. >> scan: none requested >> config: >> >> NAME STATE READ WRITE CKSUM >> zroot ONLINE 0 0 0 >> gpt/disk0 ONLINE 0 0 0 >> >> errors: No known data errors >> >> # zpool add -f zroot da1 >> # zpool add -f zroot da2 >> # zpool status >> pool: zroot >> state: ONLINE >> scan: none requested >> config: >> >> NAME STATE READ WRITE CKSUM >> zroot ONLINE 0 0 0 >> gpt/disk0 ONLINE 0 0 0 >> da1 ONLINE 0 0 0 >> da2 ONLINE 0 0 0 >> >> errors: No known data errors >> # reboot >> >> Then my system doesn’t boot anymore, i got the following error: >> >> gptzfsboot: error 4 lba 2038346899 >> gptzfsboot: error 4 lba 1361327267 >> /boot/config: -Dh >> >> BTX loader 1.00 BTX version is 1.02 >> Consoles: internal video/keyboard serial port >> BIOS drive A: is fd0 >> BIOS drive C: is disk0 >> BIOS drive D: is disk1 >> BIOS drive E: is disk2 >> BIOS drive F: is disk3 >> BIOS drive G: is disk4 >> BIOS drive H: is disk5 >> ZFS: i/o error - all block copies unavailable >> ZFS: failed to read pool zroot directory object >> BIOS 638kB/3143616kB available memory >> >> FreeBSD/x86 bootstrap loader, Revision 1.1 >> ERROR: cannot open /boot/lua/loader.lua: invalid argument. >> >> Type '?' for list of commands, 'help' for more datailed help. >> OK >> >> I can import my pool with no problems using the lived, but I could not fix it. >> >> Seems a bug after 12.1-STABLE. Please, anyone can take a look ok that? >> >> Thanks. >> >> >> >> > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Wed Oct 21 20:07:04 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 50C2542BFB8 for ; Wed, 21 Oct 2020 20:07:04 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CGhNR05Gpz3Tm9 for ; Wed, 21 Oct 2020 20:07:02 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: by mail-ej1-x629.google.com with SMTP id a3so5009962ejy.11 for ; Wed, 21 Oct 2020 13:07:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=YjTbUiP406QchyWTEtBSj4I0i1R9gYMiepUBU2/iLTA=; b=rzQmisBH5oe26K8orgGU1ivZE1AD0T5bl6zhrT8cSkKOdSCMqXp1cXrnzBWFPqZZ7I lZtDdCOr63/XrR/0pqsF1YoQwkfU+IlVAZuNK3E6P9AIP3cUIms4Inh90l4DV64Ka9Dt Hz9Xzl7m0egMR+diDi2w4frL7SqJYozttAgVoQrOXuPqZ71c9VBx7QZcAJ/kHkGuKV5R A5aR+fEkZLGr4ISWAq6NG6hZg/gQh/emPgSYf/rQo1gZqbgWDdgjt40qlHqLH2EHN+C1 W7XAIDkAZuWZ1dnsUBrfFB5KaF861kHWambnfIKKrLnsvmzb35Fa8ji1xF3itf/dW6Q7 Idnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=YjTbUiP406QchyWTEtBSj4I0i1R9gYMiepUBU2/iLTA=; b=ZprB9AYufw1Fs1SVdT29QUPLbHdWY43qJM1BzTRWNbuaCukVrhcafXLIElKgWUoxGi zt2eWKwTpKqfWMZF3ow5jcUNjyQkdNylXGqmYmAy7ModltrUIurFnvaGJStMmhKo1B/r BFBpXzIX+RQfXsFfe0eVVCjQfJRRf+QleAJe25SRpEGkVpViNkYhIJ1Vj5B/Utn9FFW8 w/6U446zI+0NBZ90Y12CYwDmUlAbiVOyzd5HTWO73l0hiCkF+AhJpcZeC91nJNNabqhb qcR3pXGZjyyOcJnFmqYdTjqWnPNZiWkbm2/uyLlZ3wu4zqgjEW+z4qivxAQBhsVKdTDZ dm0g== X-Gm-Message-State: AOAM5300mV13j1TFutpAH97h+Jz+/DOWc4Kbcd3QL3Mse3KT604u7lSp DzaC3WRnuBtBb5d/0PX+LOqXAGr79lEseVKBh/0m7ERz X-Google-Smtp-Source: ABdhPJwTQMG/3ladhzVn424kmhN2/4VLBh0l4NMoCgQrOoF5jeZItceLSrGCITyDTpshnHchej+uV+VSEJe0ZL8Fjk8= X-Received: by 2002:a17:906:cf9d:: with SMTP id um29mr5075538ejb.307.1603310821088; Wed, 21 Oct 2020 13:07:01 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Cassiano Peixoto Date: Wed, 21 Oct 2020 17:06:49 -0300 Message-ID: Subject: Re: Zpool doesn't boot anymore after FreeBSD 12.1 To: mike tancsa Cc: freebsd-stable@freebsd.org X-Rspamd-Queue-Id: 4CGhNR05Gpz3Tm9 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=rzQmisBH; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of peixotocassiano@gmail.com designates 2a00:1450:4864:20::629 as permitted sender) smtp.mailfrom=peixotocassiano@gmail.com X-Spamd-Result: default: False [-3.82 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.01)[-1.009]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.004]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::629:from]; NEURAL_HAM_SHORT(-0.81)[-0.809]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2020 20:07:04 -0000 Hi guys, Thank your for your answer. @Ricchard First of all I didn't have a chance to run zpool upgrade, because after the system update reboot i ran into the issue. @Walter and @mike Regarding making a partition, I never see any recommendation about this, I've been always using the entire disk on a pool to expand it. Until FreeBSD 11.2 it worked fine. Is there a workaround for this or should I reinstall the entire pool? On Wed, Oct 21, 2020 at 4:45 PM mike tancsa wrote: > Just a guess, Is your VM still trying to boot from whatever gpt/disk0 is > ? Or is it perhaps trying to boot from da1 or da2 which does not have > boot info ? Generally its not recommended to use the entire disk as > part of a pool. Create a partition scheme first > > gpart create -s gpt da1 > > gpart add -t freebsd-zfs da1 > > then add da1p1 to the pool > > ---Mike > > > On 10/21/2020 8:20 AM, Cassiano Peixoto wrote: > > Hi there, > > > > Anyone can help please? I've many servers with this same issue. Thanks > > > > On Fri, Oct 16, 2020 at 10:24 AM Cassiano Peixoto < > peixotocassiano@gmail.com> > > wrote: > > > >> Hi there, > >> > >> I have a FreeBSD 12.1-STABLE running on VMWARE with one disk. Then I > added two more disks to expand my pool. BTW I already did it many time wi= th > no issues. > >> > >> I ran: > >> > >> # zpool status > >> pool: zroot > >> state: ONLINE > >> status: Some supported features are not enabled on the pool. The pool > can > >> still be used, but some features are unavailable. > >> action: Enable all features using 'zpool upgrade'. Once this is done, > >> the pool may no longer be accessible by software that does not > support > >> the features. See zpool-features(7) for details. > >> scan: none requested > >> config: > >> > >> NAME STATE READ WRITE CKSUM > >> zroot ONLINE 0 0 0 > >> gpt/disk0 ONLINE 0 0 0 > >> > >> errors: No known data errors > >> > >> # zpool add -f zroot da1 > >> # zpool add -f zroot da2 > >> # zpool status > >> pool: zroot > >> state: ONLINE > >> scan: none requested > >> config: > >> > >> NAME STATE READ WRITE CKSUM > >> zroot ONLINE 0 0 0 > >> gpt/disk0 ONLINE 0 0 0 > >> da1 ONLINE 0 0 0 > >> da2 ONLINE 0 0 0 > >> > >> errors: No known data errors > >> # reboot > >> > >> Then my system doesn=E2=80=99t boot anymore, i got the following error= : > >> > >> gptzfsboot: error 4 lba 2038346899 > >> gptzfsboot: error 4 lba 1361327267 > >> /boot/config: -Dh > >> > >> BTX loader 1.00 BTX version is 1.02 > >> Consoles: internal video/keyboard serial port > >> BIOS drive A: is fd0 > >> BIOS drive C: is disk0 > >> BIOS drive D: is disk1 > >> BIOS drive E: is disk2 > >> BIOS drive F: is disk3 > >> BIOS drive G: is disk4 > >> BIOS drive H: is disk5 > >> ZFS: i/o error - all block copies unavailable > >> ZFS: failed to read pool zroot directory object > >> BIOS 638kB/3143616kB available memory > >> > >> FreeBSD/x86 bootstrap loader, Revision 1.1 > >> ERROR: cannot open /boot/lua/loader.lua: invalid argument. > >> > >> Type '?' for list of commands, 'help' for more datailed help. > >> OK > >> > >> I can import my pool with no problems using the lived, but I could not > fix it. > >> > >> Seems a bug after 12.1-STABLE. Please, anyone can take a look ok that? > >> > >> Thanks. > >> > >> > >> > >> > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.or= g > " > > > From owner-freebsd-stable@freebsd.org Wed Oct 21 20:18:51 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 829C642C648 for ; Wed, 21 Oct 2020 20:18:51 +0000 (UTC) (envelope-from wfc@mintsol.com) Received: from scully.mintsol.com (scully.mintsol.com [199.182.77.206]) by mx1.freebsd.org (Postfix) with ESMTP id 4CGhf24TXdz3Tp6 for ; Wed, 21 Oct 2020 20:18:50 +0000 (UTC) (envelope-from wfc@mintsol.com) Received: from mintsol.com (officecc.mintsol.com [96.85.114.33]) by scully.mintsol.com with esmtp; Wed, 21 Oct 2020 16:18:43 -0400 id 00B12C39.000000005F9097A3.00009B29 Received: from localhost (localhost [127.0.0.1]) (IDENT: uid 1002) by mintsol.com with esmtp; Wed, 21 Oct 2020 16:18:43 -0400 id 00000B10.5F9097A3.00003F85 Date: Wed, 21 Oct 2020 16:18:43 -0400 (EDT) From: Walter Cramer To: Cassiano Peixoto cc: mike tancsa , freebsd-stable@freebsd.org Subject: Re: Zpool doesn't boot anymore after FreeBSD 12.1 In-Reply-To: Message-ID: <20201021160923.T14777@mulder.mintsol.com> References: MIME-Version: 1.0 Content-ID: <20201021161636.X14777@mulder.mintsol.com> X-Rspamd-Queue-Id: 4CGhf24TXdz3Tp6 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of wfc@mintsol.com designates 199.182.77.206 as permitted sender) smtp.mailfrom=wfc@mintsol.com X-Spamd-Result: default: False [-1.20 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.89)[-0.892]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:scully.mintsol.com]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; DMARC_NA(0.00)[mintsol.com]; NEURAL_HAM_LONG(-1.00)[-0.998]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.11)[-0.114]; CTYPE_MIXED_BOGUS(1.00)[]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:22768, ipnet:199.182.77.0/24, country:US]; MAILMAN_DEST(0.00)[freebsd-stable] Content-Type: TEXT/PLAIN; CHARSET=X-UNKNOWN; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2020 20:18:51 -0000 My guess - there is a work-around or two, but you'll face a lot more=20 grief, long-term, if you don't do things the right way (aka do a bunch of= =20 re-install work) now. I'd start with 'gpart backup da0' (guessing that gpt/disk0 is on da0), to= =20 see how the original disk is partitioned. Then duplicate that=20 partitioning on da1 and da2. -Walter On Wed, 21 Oct 2020, Cassiano Peixoto wrote: > Hi guys, Thank your for your answer. @Ricchard First of all I didn't have a chance to run zpool upgrade, because after the system update reboot i ran into the issue. @Walter and @mike Regarding making a partition, I never see any recommendation about this, I've been always using the entire disk on a pool to expand it. Until FreeBSD 11.2 it worked fine. Is there a workaround for this or should I reinstall the entire pool? On Wed, Oct 21, 2020 at 4:45 PM mike tancsa wrote: > Just a guess, Is your VM still trying to boot from whatever gpt/disk0 is > ? Or is it perhaps trying to boot from da1 or da2 which does not have > boot info ? Generally its not recommended to use the entire disk as > part of a pool. Create a partition scheme first > > gpart create -s gpt da1 > > gpart add -t freebsd-zfs da1 > > then add da1p1 to the pool > > ---Mike > > > On 10/21/2020 8:20 AM, Cassiano Peixoto wrote: > > Hi there, > > > > Anyone can help please? I've many servers with this same issue. Thanks > > > > On Fri, Oct 16, 2020 at 10:24 AM Cassiano Peixoto < > peixotocassiano@gmail.com> > > wrote: > > > >> Hi there, > >> > >> I have a FreeBSD 12.1-STABLE running on VMWARE with one disk. Then I > added two more disks to expand my pool. BTW I already did it many time wi= th > no issues. > >> > >> I ran: > >> > >> # zpool status > >> pool: zroot > >> state: ONLINE > >> status: Some supported features are not enabled on the pool. The pool > can > >> still be used, but some features are unavailable. > >> action: Enable all features using 'zpool upgrade'. Once this is done, > >> the pool may no longer be accessible by software that does not > support > >> the features. See zpool-features(7) for details. > >> scan: none requested > >> config: > >> > >> NAME STATE READ WRITE CKSUM > >> zroot ONLINE 0 0 0 > >> gpt/disk0 ONLINE 0 0 0 > >> > >> errors: No known data errors > >> > >> # zpool add -f zroot da1 > >> # zpool add -f zroot da2 > >> # zpool status > >> pool: zroot > >> state: ONLINE > >> scan: none requested > >> config: > >> > >> NAME STATE READ WRITE CKSUM > >> zroot ONLINE 0 0 0 > >> gpt/disk0 ONLINE 0 0 0 > >> da1 ONLINE 0 0 0 > >> da2 ONLINE 0 0 0 > >> > >> errors: No known data errors > >> # reboot > >> > >> Then my system doesn=E2=80=99t boot anymore, i got the following error= : > >> > >> gptzfsboot: error 4 lba 2038346899 > >> gptzfsboot: error 4 lba 1361327267 > >> /boot/config: -Dh > >> > >> BTX loader 1.00 BTX version is 1.02 > >> Consoles: internal video/keyboard serial port > >> BIOS drive A: is fd0 > >> BIOS drive C: is disk0 > >> BIOS drive D: is disk1 > >> BIOS drive E: is disk2 > >> BIOS drive F: is disk3 > >> BIOS drive G: is disk4 > >> BIOS drive H: is disk5 > >> ZFS: i/o error - all block copies unavailable > >> ZFS: failed to read pool zroot directory object > >> BIOS 638kB/3143616kB available memory > >> > >> FreeBSD/x86 bootstrap loader, Revision 1.1 > >> ERROR: cannot open /boot/lua/loader.lua: invalid argument. > >> > >> Type '?' for list of commands, 'help' for more datailed help. > >> OK > >> > >> I can import my pool with no problems using the lived, but I could not > fix it. > >> > >> Seems a bug after 12.1-STABLE. Please, anyone can take a look ok that? > >> > >> Thanks. > >> > >> > >> > >> > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.or= g > " > > > _______________________________________________ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Wed Oct 21 20:24:19 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3FFDC42CAB8 for ; Wed, 21 Oct 2020 20:24:19 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CGhmL1zt4z3VZh for ; Wed, 21 Oct 2020 20:24:18 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: by mail-ej1-x62b.google.com with SMTP id t25so5055798ejd.13 for ; Wed, 21 Oct 2020 13:24:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=bYM8uyZRxakBUD7crU0t4qRaO27VzD3XSrbRx5qtPmw=; b=fbRhC8/FRj9Kp43NdAFlTVFo2wXgOlrrXyT4/ozKLXN6tftacPBOP83zaI9r3YIZYV cHmP0YpWNj15yrygzMbZ7uaKMFGKARqE1tcXfzkG5RcQfcCEXuMEW4yinzALO7HD9j3J F2YZLkADiVu4lg0mwBUgaVfdi7RcvYxKIB/McxBFCcVyVNJ5/gWDJx78BRu667pag5Mm 8oO8XTS/SFIbiu5UrwUCjvHAzJzLYH2VMHazWJ0rZvSMbw7DY2sirRF6Ue1Ruzxe+4gY /s7HUKk0cWSemWSgjMdjJnptT72E0L/voFTFcuGSZ1hG7ZGz7A6Dshcvd0kczk6rQ0me /Rxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=bYM8uyZRxakBUD7crU0t4qRaO27VzD3XSrbRx5qtPmw=; b=TucRmzPV8+PpLahT4uiicIpIFhECsrSoxJky3ne6gJ4KTmVkb8bBCmet4/paW9nYd3 qe/JyK+Vg7Wt6mh640XtKHqRKJDH7yxw14Y1TYiqIGMN6rrE6ntfus8JpflXDcgM/aU2 0C6r2Qz9C5VfDmk50OitHmA8P/k1D00vQtGT7Oe3XfXAoDlpUwMmPE41/hwEfPUseYDB IL0rJpAheJcCKGKTL7Vj9OpoAjp2tsgSj06mRQDhRG7rugngbwvuvwtx03X9EWObv0GR 7AQ2PB88dwok1UKNnTYdYxBprBL5JC3EZR+/7qTpCR0k2r748f0pyQCQlIwRTfbWGKj8 rEUw== X-Gm-Message-State: AOAM532o8G/J8EHYFpvT79fkpWjv5toC9W4KmH5FFQB4oAsJOsqhXC4V Lkd5vtBEr1BGpfWc/LfHxflmFRrfxQMJdjnOxHU= X-Google-Smtp-Source: ABdhPJyIx7qGABQI/0vwTeEtDgVednbPYGdm318WgDOy/a+c86wxeCfLSwQhrdhYNaStZiiKb3hWa7rroP2I0uDIOBA= X-Received: by 2002:a17:906:9483:: with SMTP id t3mr5546135ejx.390.1603311856717; Wed, 21 Oct 2020 13:24:16 -0700 (PDT) MIME-Version: 1.0 References: <20201021160923.T14777@mulder.mintsol.com> In-Reply-To: <20201021160923.T14777@mulder.mintsol.com> From: Cassiano Peixoto Date: Wed, 21 Oct 2020 17:24:05 -0300 Message-ID: Subject: Re: Zpool doesn't boot anymore after FreeBSD 12.1 To: Walter Cramer Cc: mike tancsa , freebsd-stable@freebsd.org X-Rspamd-Queue-Id: 4CGhmL1zt4z3VZh X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=fbRhC8/F; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of peixotocassiano@gmail.com designates 2a00:1450:4864:20::62b as permitted sender) smtp.mailfrom=peixotocassiano@gmail.com X-Spamd-Result: default: False [-3.79 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.01)[-1.006]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_HAM_LONG(-1.01)[-1.010]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62b:from]; NEURAL_HAM_SHORT(-0.77)[-0.774]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2020 20:24:19 -0000 Walter, Yes, gpt/disk0 is da0. I can do the partition backup booting from livecd. But is there a way to make the partition of da1 and da2 since both are already inserted on pool? I think it's not allowed... Thanks. On Wed, Oct 21, 2020 at 5:18 PM Walter Cramer wrote: > My guess - there is a work-around or two, but you'll face a lot more > grief, long-term, if you don't do things the right way (aka do a bunch of > re-install work) now. > > I'd start with 'gpart backup da0' (guessing that gpt/disk0 is on da0), to > see how the original disk is partitioned. Then duplicate that > partitioning on da1 and da2. > > -Walter > > On Wed, 21 Oct 2020, Cassiano Peixoto wrote: > > > Hi guys, > > Thank your for your answer. > > @Ricchard First of all I didn't have a chance to run zpool upgrade, becau= se > after the system update reboot i ran into the issue. > > @Walter and @mike Regarding making a partition, I never see any > recommendation about this, I've been always using the entire disk on a po= ol > to expand it. Until FreeBSD 11.2 it worked fine. > > Is there a workaround for this or should I reinstall the entire pool? > > On Wed, Oct 21, 2020 at 4:45 PM mike tancsa wrote: > > > Just a guess, Is your VM still trying to boot from whatever gpt/disk0 i= s > > ? Or is it perhaps trying to boot from da1 or da2 which does not have > > boot info ? Generally its not recommended to use the entire disk as > > part of a pool. Create a partition scheme first > > > > gpart create -s gpt da1 > > > > gpart add -t freebsd-zfs da1 > > > > then add da1p1 to the pool > > > > ---Mike > > > > > > On 10/21/2020 8:20 AM, Cassiano Peixoto wrote: > > > Hi there, > > > > > > Anyone can help please? I've many servers with this same issue. Thank= s > > > > > > On Fri, Oct 16, 2020 at 10:24 AM Cassiano Peixoto < > > peixotocassiano@gmail.com> > > > wrote: > > > > > >> Hi there, > > >> > > >> I have a FreeBSD 12.1-STABLE running on VMWARE with one disk. Then I > > added two more disks to expand my pool. BTW I already did it many time > with > > no issues. > > >> > > >> I ran: > > >> > > >> # zpool status > > >> pool: zroot > > >> state: ONLINE > > >> status: Some supported features are not enabled on the pool. The poo= l > > can > > >> still be used, but some features are unavailable. > > >> action: Enable all features using 'zpool upgrade'. Once this is done= , > > >> the pool may no longer be accessible by software that does not > > support > > >> the features. See zpool-features(7) for details. > > >> scan: none requested > > >> config: > > >> > > >> NAME STATE READ WRITE CKSUM > > >> zroot ONLINE 0 0 0 > > >> gpt/disk0 ONLINE 0 0 0 > > >> > > >> errors: No known data errors > > >> > > >> # zpool add -f zroot da1 > > >> # zpool add -f zroot da2 > > >> # zpool status > > >> pool: zroot > > >> state: ONLINE > > >> scan: none requested > > >> config: > > >> > > >> NAME STATE READ WRITE CKSUM > > >> zroot ONLINE 0 0 0 > > >> gpt/disk0 ONLINE 0 0 0 > > >> da1 ONLINE 0 0 0 > > >> da2 ONLINE 0 0 0 > > >> > > >> errors: No known data errors > > >> # reboot > > >> > > >> Then my system doesn=E2=80=99t boot anymore, i got the following err= or: > > >> > > >> gptzfsboot: error 4 lba 2038346899 > > >> gptzfsboot: error 4 lba 1361327267 > > >> /boot/config: -Dh > > >> > > >> BTX loader 1.00 BTX version is 1.02 > > >> Consoles: internal video/keyboard serial port > > >> BIOS drive A: is fd0 > > >> BIOS drive C: is disk0 > > >> BIOS drive D: is disk1 > > >> BIOS drive E: is disk2 > > >> BIOS drive F: is disk3 > > >> BIOS drive G: is disk4 > > >> BIOS drive H: is disk5 > > >> ZFS: i/o error - all block copies unavailable > > >> ZFS: failed to read pool zroot directory object > > >> BIOS 638kB/3143616kB available memory > > >> > > >> FreeBSD/x86 bootstrap loader, Revision 1.1 > > >> ERROR: cannot open /boot/lua/loader.lua: invalid argument. > > >> > > >> Type '?' for list of commands, 'help' for more datailed help. > > >> OK > > >> > > >> I can import my pool with no problems using the lived, but I could n= ot > > fix it. > > >> > > >> Seems a bug after 12.1-STABLE. Please, anyone can take a look ok tha= t? > > >> > > >> Thanks. > > >> > > >> > > >> > > >> > > > _______________________________________________ > > > freebsd-stable@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > > > To unsubscribe, send any mail to " > freebsd-stable-unsubscribe@freebsd.org > > " > > > > > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Wed Oct 21 20:36:19 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C12AB42D324 for ; Wed, 21 Oct 2020 20:36:19 +0000 (UTC) (envelope-from wfc@mintsol.com) Received: from scully.mintsol.com (scully.mintsol.com [199.182.77.206]) by mx1.freebsd.org (Postfix) with ESMTP id 4CGj2B5PyQz3WQD for ; Wed, 21 Oct 2020 20:36:18 +0000 (UTC) (envelope-from wfc@mintsol.com) Received: from mintsol.com (officecc.mintsol.com [96.85.114.33]) by scully.mintsol.com with esmtp; Wed, 21 Oct 2020 16:36:17 -0400 id 00B12C91.000000005F909BC2.0000B757 Received: from localhost (localhost [127.0.0.1]) (IDENT: uid 1002) by mintsol.com with esmtp; Wed, 21 Oct 2020 16:36:17 -0400 id 00000B88.5F909BC1.00004285 Date: Wed, 21 Oct 2020 16:36:17 -0400 (EDT) From: Walter Cramer To: Cassiano Peixoto cc: freebsd-stable@freebsd.org Subject: Re: Zpool doesn't boot anymore after FreeBSD 12.1 In-Reply-To: Message-ID: <20201021162554.K14777@mulder.mintsol.com> References: <20201021160923.T14777@mulder.mintsol.com> MIME-Version: 1.0 Content-ID: <20201021163120.N14777@mulder.mintsol.com> X-Rspamd-Queue-Id: 4CGj2B5PyQz3WQD X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of wfc@mintsol.com designates 199.182.77.206 as permitted sender) smtp.mailfrom=wfc@mintsol.com X-Spamd-Result: default: False [-1.26 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.92)[-0.922]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:scully.mintsol.com]; NEURAL_HAM_LONG(-0.99)[-0.991]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; DMARC_NA(0.00)[mintsol.com]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.15)[-0.152]; CTYPE_MIXED_BOGUS(1.00)[]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:22768, ipnet:199.182.77.0/24, country:US]; MAILMAN_DEST(0.00)[freebsd-stable] Content-Type: TEXT/PLAIN; CHARSET=X-UNKNOWN; FORMAT=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2020 20:36:19 -0000 I very much doubt that there's any remotely sane way to re-partition da1 &= =20 da2 while they are in the pool. If you carefully edit the output of the 'gpart backup' command, you=20 probably can use 'gpart restore' to partition da1 and da2. Read the=20 'gpart' man page very carefully, if you aren't familiar with that. Also,= =20 you'll probably want to install bootstrap code on da1 & da2 with 'gpart=20 bootstrap'. -Walter On Wed, 21 Oct 2020, Cassiano Peixoto wrote: > Walter, Yes, gpt/disk0 is da0. I can do the partition backup booting from livecd. But is there a way to make the partition of da1 and da2 since both are already inserted on pool? I think it's not allowed... Thanks. On Wed, Oct 21, 2020 at 5:18 PM Walter Cramer wrote: > My guess - there is a work-around or two, but you'll face a lot more > grief, long-term, if you don't do things the right way (aka do a bunch of > re-install work) now. > > I'd start with 'gpart backup da0' (guessing that gpt/disk0 is on da0), to > see how the original disk is partitioned. Then duplicate that > partitioning on da1 and da2. > > -Walter > > On Wed, 21 Oct 2020, Cassiano Peixoto wrote: > > > Hi guys, > > Thank your for your answer. > > @Ricchard First of all I didn't have a chance to run zpool upgrade, becau= se > after the system update reboot i ran into the issue. > > @Walter and @mike Regarding making a partition, I never see any > recommendation about this, I've been always using the entire disk on a po= ol > to expand it. Until FreeBSD 11.2 it worked fine. > > Is there a workaround for this or should I reinstall the entire pool? > > On Wed, Oct 21, 2020 at 4:45 PM mike tancsa wrote: > > > Just a guess, Is your VM still trying to boot from whatever gpt/disk0 i= s > > ? Or is it perhaps trying to boot from da1 or da2 which does not have > > boot info ? Generally its not recommended to use the entire disk as > > part of a pool. Create a partition scheme first > > > > gpart create -s gpt da1 > > > > gpart add -t freebsd-zfs da1 > > > > then add da1p1 to the pool > > > > ---Mike > > > > > > On 10/21/2020 8:20 AM, Cassiano Peixoto wrote: > > > Hi there, > > > > > > Anyone can help please? I've many servers with this same issue. Thank= s > > > > > > On Fri, Oct 16, 2020 at 10:24 AM Cassiano Peixoto < > > peixotocassiano@gmail.com> > > > wrote: > > > > > >> Hi there, > > >> > > >> I have a FreeBSD 12.1-STABLE running on VMWARE with one disk. Then I > > added two more disks to expand my pool. BTW I already did it many time > with > > no issues. > > >> > > >> I ran: > > >> > > >> # zpool status > > >> pool: zroot > > >> state: ONLINE > > >> status: Some supported features are not enabled on the pool. The poo= l > > can > > >> still be used, but some features are unavailable. > > >> action: Enable all features using 'zpool upgrade'. Once this is done= , > > >> the pool may no longer be accessible by software that does not > > support > > >> the features. See zpool-features(7) for details. > > >> scan: none requested > > >> config: > > >> > > >> NAME STATE READ WRITE CKSUM > > >> zroot ONLINE 0 0 0 > > >> gpt/disk0 ONLINE 0 0 0 > > >> > > >> errors: No known data errors > > >> > > >> # zpool add -f zroot da1 > > >> # zpool add -f zroot da2 > > >> # zpool status > > >> pool: zroot > > >> state: ONLINE > > >> scan: none requested > > >> config: > > >> > > >> NAME STATE READ WRITE CKSUM > > >> zroot ONLINE 0 0 0 > > >> gpt/disk0 ONLINE 0 0 0 > > >> da1 ONLINE 0 0 0 > > >> da2 ONLINE 0 0 0 > > >> > > >> errors: No known data errors > > >> # reboot > > >> > > >> Then my system doesn=E2=80=99t boot anymore, i got the following err= or: > > >> > > >> gptzfsboot: error 4 lba 2038346899 > > >> gptzfsboot: error 4 lba 1361327267 > > >> /boot/config: -Dh > > >> > > >> BTX loader 1.00 BTX version is 1.02 > > >> Consoles: internal video/keyboard serial port > > >> BIOS drive A: is fd0 > > >> BIOS drive C: is disk0 > > >> BIOS drive D: is disk1 > > >> BIOS drive E: is disk2 > > >> BIOS drive F: is disk3 > > >> BIOS drive G: is disk4 > > >> BIOS drive H: is disk5 > > >> ZFS: i/o error - all block copies unavailable > > >> ZFS: failed to read pool zroot directory object > > >> BIOS 638kB/3143616kB available memory > > >> > > >> FreeBSD/x86 bootstrap loader, Revision 1.1 > > >> ERROR: cannot open /boot/lua/loader.lua: invalid argument. > > >> > > >> Type '?' for list of commands, 'help' for more datailed help. > > >> OK > > >> > > >> I can import my pool with no problems using the lived, but I could n= ot > > fix it. > > >> > > >> Seems a bug after 12.1-STABLE. Please, anyone can take a look ok tha= t? > > >> > > >> Thanks. > > >> > > >> > > >> > > >> > > > _______________________________________________ > > > freebsd-stable@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > > > To unsubscribe, send any mail to " > freebsd-stable-unsubscribe@freebsd.org > > " > > > > > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" _______________________________________________ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Wed Oct 21 21:50:46 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E6F2042E9A6 for ; Wed, 21 Oct 2020 21:50:46 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [IPv6:2a00:1450:4864:20::52c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CGkh600hZz3ZcF for ; Wed, 21 Oct 2020 21:50:45 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: by mail-ed1-x52c.google.com with SMTP id t20so4043067edr.11 for ; Wed, 21 Oct 2020 14:50:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=M0PcM0UELISwSLrr/9zSNpqPEhGMsMgb+jNX8AbXYZ0=; b=mRREEDuu7HmoUppzumxG5OWdWzyaQTrKcEkBK7MBjtGynRpgJGcPyfOPxY/0sAoqT5 vFR8nmhfGU5lfPnvCrcsvYGjH+rNb1dCh5nMWy/Bc/dCyG1/qsiH6DZBQhVW4M5AqsrM TbRv4QPuq3/CZiDsggFD3RkmvDhahqesJgvjGZd+SjcZLP3L8yAbKEobkx24Sah3kGN4 HktHUedb+KPVKNhAxcK5W0UpssNLGy5IvbD6/mWOd2PrKBzwGLXizzf3c5DlkJScpXpW xBnKvJj9meMtnAUZcCtYUsPvVQq7MFfwHsSI9EJ0w5WAyJ5lsc0iucGPV/Kk+8d62JvP m7wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=M0PcM0UELISwSLrr/9zSNpqPEhGMsMgb+jNX8AbXYZ0=; b=LYH4b6VdkOOxgk74POJ7OvnNigIXMvVNsnXaYkzdTkF03i1c0B0hSsqSQ7Y2TwjmM5 AUib+PxMN96/+iMvryTsZudJaDhezOIR27CrzsLHLOCWbxU5dcdziz8h88Mf6rzetOvw ACMJmiOfG701XeEbDN+PdQ2NIYrtggK31HBPw00DyoQYD+YEGadBND4shLpHGWGCQdTV i+4h4vwVDCDJZRFxHG0fjF8J46n5qzJOMmaXMndzan++cpDUkoHk7WYW7Hbc280a6Yel 66NcR6OLG+XdB9Mp6+KpZQAHg6XPHPx5IcsWOfG48Jwjgs9yejR4o5qkBWpykkCCx+C5 nfjg== X-Gm-Message-State: AOAM5303xZDsvElisrB0QKjCTDMnh+Y7c5f5/CMne2J7EogTiRjv8Jtv Dt3S8D8CjDYE/qjXITH34ReS5JfxqjcMIua4Zhg= X-Google-Smtp-Source: ABdhPJyU1T6wBB9So5SlzI1mYwZIZQxNapHE73D8o8YFNAgv27fQ18zNAhZ9wf3vqC3HTLHONimWYhx+HKzV0LKa9Ro= X-Received: by 2002:a05:6402:744:: with SMTP id p4mr4969539edy.190.1603317044291; Wed, 21 Oct 2020 14:50:44 -0700 (PDT) MIME-Version: 1.0 References: <20201021160923.T14777@mulder.mintsol.com> <20201021162554.K14777@mulder.mintsol.com> In-Reply-To: <20201021162554.K14777@mulder.mintsol.com> From: Cassiano Peixoto Date: Wed, 21 Oct 2020 18:50:32 -0300 Message-ID: Subject: Re: Zpool doesn't boot anymore after FreeBSD 12.1 To: Walter Cramer Cc: freebsd-stable@freebsd.org X-Rspamd-Queue-Id: 4CGkh600hZz3ZcF X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=mRREEDuu; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of peixotocassiano@gmail.com designates 2a00:1450:4864:20::52c as permitted sender) smtp.mailfrom=peixotocassiano@gmail.com X-Spamd-Result: default: False [-3.53 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.01)[-1.009]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.004]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::52c:from]; NEURAL_HAM_SHORT(-0.52)[-0.519]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2020 21:50:47 -0000 Thanks for the tip! I've fixed the issue following the following steps: 1) added two new disks with the same size: da3 and da4 2) Made a partition for both: gpart create -s gpt da3 gpart create -s gpt da4 gpart add -t freebsd-zfs da3 gpart add -t freebsd-zfs da4 3) replace both disks on pool zpool replace zroot da2 da3p1 zpool replace zroot da3 da4p1 4) rebuild bootcode gpart boocode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0 Done! I rebooted and everything is working again. Thank you guys for the support. On Wed, Oct 21, 2020 at 5:36 PM Walter Cramer wrote: > I very much doubt that there's any remotely sane way to re-partition da1 = & > da2 while they are in the pool. > > If you carefully edit the output of the 'gpart backup' command, you > probably can use 'gpart restore' to partition da1 and da2. Read the > 'gpart' man page very carefully, if you aren't familiar with that. Also, > you'll probably want to install bootstrap code on da1 & da2 with 'gpart > bootstrap'. > > -Walter > > On Wed, 21 Oct 2020, Cassiano Peixoto wrote: > > > Walter, > > Yes, gpt/disk0 is da0. I can do the partition backup booting from livecd. > But is there a way to make the partition of da1 and da2 since both are > already inserted on pool? I think it's not allowed... > > Thanks. > > > On Wed, Oct 21, 2020 at 5:18 PM Walter Cramer wrote: > > > My guess - there is a work-around or two, but you'll face a lot more > > grief, long-term, if you don't do things the right way (aka do a bunch = of > > re-install work) now. > > > > I'd start with 'gpart backup da0' (guessing that gpt/disk0 is on da0), = to > > see how the original disk is partitioned. Then duplicate that > > partitioning on da1 and da2. > > > > -Walter > > > > On Wed, 21 Oct 2020, Cassiano Peixoto wrote: > > > > > Hi guys, > > > > Thank your for your answer. > > > > @Ricchard First of all I didn't have a chance to run zpool upgrade, > because > > after the system update reboot i ran into the issue. > > > > @Walter and @mike Regarding making a partition, I never see any > > recommendation about this, I've been always using the entire disk on a > pool > > to expand it. Until FreeBSD 11.2 it worked fine. > > > > Is there a workaround for this or should I reinstall the entire pool? > > > > On Wed, Oct 21, 2020 at 4:45 PM mike tancsa wrote: > > > > > Just a guess, Is your VM still trying to boot from whatever gpt/disk0 > is > > > ? Or is it perhaps trying to boot from da1 or da2 which does not have > > > boot info ? Generally its not recommended to use the entire disk as > > > part of a pool. Create a partition scheme first > > > > > > gpart create -s gpt da1 > > > > > > gpart add -t freebsd-zfs da1 > > > > > > then add da1p1 to the pool > > > > > > ---Mike > > > > > > > > > On 10/21/2020 8:20 AM, Cassiano Peixoto wrote: > > > > Hi there, > > > > > > > > Anyone can help please? I've many servers with this same issue. > Thanks > > > > > > > > On Fri, Oct 16, 2020 at 10:24 AM Cassiano Peixoto < > > > peixotocassiano@gmail.com> > > > > wrote: > > > > > > > >> Hi there, > > > >> > > > >> I have a FreeBSD 12.1-STABLE running on VMWARE with one disk. Then= I > > > added two more disks to expand my pool. BTW I already did it many tim= e > > with > > > no issues. > > > >> > > > >> I ran: > > > >> > > > >> # zpool status > > > >> pool: zroot > > > >> state: ONLINE > > > >> status: Some supported features are not enabled on the pool. The > pool > > > can > > > >> still be used, but some features are unavailable. > > > >> action: Enable all features using 'zpool upgrade'. Once this is > done, > > > >> the pool may no longer be accessible by software that does no= t > > > support > > > >> the features. See zpool-features(7) for details. > > > >> scan: none requested > > > >> config: > > > >> > > > >> NAME STATE READ WRITE CKSUM > > > >> zroot ONLINE 0 0 0 > > > >> gpt/disk0 ONLINE 0 0 0 > > > >> > > > >> errors: No known data errors > > > >> > > > >> # zpool add -f zroot da1 > > > >> # zpool add -f zroot da2 > > > >> # zpool status > > > >> pool: zroot > > > >> state: ONLINE > > > >> scan: none requested > > > >> config: > > > >> > > > >> NAME STATE READ WRITE CKSUM > > > >> zroot ONLINE 0 0 0 > > > >> gpt/disk0 ONLINE 0 0 0 > > > >> da1 ONLINE 0 0 0 > > > >> da2 ONLINE 0 0 0 > > > >> > > > >> errors: No known data errors > > > >> # reboot > > > >> > > > >> Then my system doesn=E2=80=99t boot anymore, i got the following e= rror: > > > >> > > > >> gptzfsboot: error 4 lba 2038346899 > > > >> gptzfsboot: error 4 lba 1361327267 > > > >> /boot/config: -Dh > > > >> > > > >> BTX loader 1.00 BTX version is 1.02 > > > >> Consoles: internal video/keyboard serial port > > > >> BIOS drive A: is fd0 > > > >> BIOS drive C: is disk0 > > > >> BIOS drive D: is disk1 > > > >> BIOS drive E: is disk2 > > > >> BIOS drive F: is disk3 > > > >> BIOS drive G: is disk4 > > > >> BIOS drive H: is disk5 > > > >> ZFS: i/o error - all block copies unavailable > > > >> ZFS: failed to read pool zroot directory object > > > >> BIOS 638kB/3143616kB available memory > > > >> > > > >> FreeBSD/x86 bootstrap loader, Revision 1.1 > > > >> ERROR: cannot open /boot/lua/loader.lua: invalid argument. > > > >> > > > >> Type '?' for list of commands, 'help' for more datailed help. > > > >> OK > > > >> > > > >> I can import my pool with no problems using the lived, but I could > not > > > fix it. > > > >> > > > >> Seems a bug after 12.1-STABLE. Please, anyone can take a look ok > that? > > > >> > > > >> Thanks. > > > >> > > > >> > > > >> > > > >> > > > > _______________________________________________ > > > > freebsd-stable@freebsd.org mailing list > > > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > > > > To unsubscribe, send any mail to " > > freebsd-stable-unsubscribe@freebsd.org > > > " > > > > > > > > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.or= g > " > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Wed Oct 21 22:07:23 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 02B5642FAA7 for ; Wed, 21 Oct 2020 22:07:23 +0000 (UTC) (envelope-from debdrup@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4CGl3G5fDNz3cDl for ; Wed, 21 Oct 2020 22:07:22 +0000 (UTC) (envelope-from debdrup@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id B804A42FAA4; Wed, 21 Oct 2020 22:07:22 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B69F842FB1D; Wed, 21 Oct 2020 22:07:22 +0000 (UTC) (envelope-from debdrup@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CGl3G4mnNz3cGj; Wed, 21 Oct 2020 22:07:22 +0000 (UTC) (envelope-from debdrup@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1603318042; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=3k/7cOhY3ykWXAyGhMTeKCk5//0GlnJUC6835QYUvCM=; b=JYou8eONS62AFyGj+y1ES3C0NJGsYC2mc6IZIjxVnb6QMN0De1r5sL0yBYfiqpcUBRIUl4 KYN/C4eUvS2LxTkrY+ne4YtqgR05H51jons9avgonDASFEEgCbHWqpepKgG8cmEfohtyWI 2CD5MBt6lxTFLrcOCvTAWVBtbKGKRevG2CkzakXj9ZREaDa/6RVNyAdGebnXWP69z5Y4Lg 6fx4MwP0cRwSyQ6vpFYCHOivz18eFuM9mx4fJgnOWPSC3eAJHgpm2d2D9QbLrNX1yfmA1L kuG8ZhdUCPd15AkoRCEsalTyExFVodyVJWzEl7nJ93OtwnMGZ5LRPb9zfvFw0w== Received: by freefall.freebsd.org (Postfix, from userid 1471) id 9593F1955C; Wed, 21 Oct 2020 22:07:22 +0000 (UTC) Date: Thu, 22 Oct 2020 00:07:20 +0200 From: Daniel Ebdrup Jensen To: hackers@freebsd.org Cc: current@freebsd.org, stable@freebsd.org Subject: FreeBSD Quarterly Status Report - Third Quarter 2020 Message-ID: <20201021220720.mehxu5cjqcysztzt@nerd-thinkpad.local> Mail-Followup-To: Daniel Ebdrup Jensen , hackers@freebsd.org, current@freebsd.org, stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="u3hngvn2nte45ec7" Content-Disposition: inline ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1603318042; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=3k/7cOhY3ykWXAyGhMTeKCk5//0GlnJUC6835QYUvCM=; b=tV4iX0Bh9k9knz+fYfW+3x38mBPBxj/JzDj+DL+rg5qSkthXXMM6CuQSpskV138n2micq0 Orxnw4seKGmsVBDazgg2g0pkeL9aFbSsNf8lVXx8zEW6/ZIdn/NUH+KveAAndfRer0jTry EX8KcxHaJ9RWU1doab+ZthtX72E9Kf48oDxiqiIyDRcp/SOuQnknPvpx3HJToUGTtQ/xgd Psqace19HJHcZ3LO0vlNF74fcYvT9AU+GpcLdHHalVo6++OaUwAO1cITLjkCZLVK6V+Q+a u6T3Kda9a+BNoSKq3CC9ZOrBai01C5BqI6KRegPf+zwAjdxww/LOPDRo2opH8Q== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1603318042; a=rsa-sha256; cv=none; b=J5KDIiI0oNlp9pCfuZG9KDMUnxdRqC93yyV7S5eAEzBvRnFwnwaEgBGBxCbDl9WHw2mU6e dHHDA2VSbDeG2vmvNQEZ4ppecfili8F+9ek7XtrFOB2kkuGGBwkFLfzGF5zj0QIShIm0hF MuHYtAbSmS+o7D91yU3Td2rIfDi0krTOrpM0nrO0y/F/fBBW7OgHPORgo3aMQeb+abx7cY 0tN7z/4kR2UHpxWu96St1GDe7K1faUsF30A7K5lGoei6kulk7Q/5VTNhUZPqmR1XDDcvqO Ij0Ymk5pBrLWV4uVe67nsKP1+9v1mcvjDHg/7TgyN+n8mEE+zYbs2on+fq6kCQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2020 22:07:23 -0000 --u3hngvn2nte45ec7 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable FreeBSD Project Quarterly Status Report - Third Quarter 2020 Introduction This report covers FreeBSD related projects for the period between July and September, and is the third of four planned reports for 2020. This quarter brings a good mix of additions and changes to the FreeBSD Project and community, from a diverse number of teams and people covering everything from architectures, continuous integration, wireless networking and drivers, over drm, desktop and third-party project work, as well as several team reports, along with many other interesting subjects too numerous to mention. As the world is still affected by the epidemic, we hope that this report can also serve as a good reminder that there is good work that can be done by people working together, even if we're apart. We hope you'll be as interested in reading it, as we've been in making it. Daniel Ebdrup Jensen, on behalf of the quarterly team. __________________________________________________________________ FreeBSD Team Reports * FreeBSD Foundation * FreeBSD Release Engineering Team * Cluster Administration Team * Continuous Integration * Ports Collection * FreeBSD Office team - 3rd quarter 2020 report * FreeBSD Graphics Team status report Projects * FreeBSD on Microsoft HyperV and Azure * Building FreeBSD on non-FreeBSD hosts * Git Migration Working Group * Linux compatibility layer update * LLDB Debugger Improvements * Lua usage in FreeBSD * NFS over TLS implementation * syzkaller on FreeBSD Kernel * DRM Drivers Update * DTS Update * DesignWare Ethernet adapter driver improvements * Google Summer of Code'20 Project - eBPF XDP Hooks * ENA FreeBSD Driver Update * IPSec Extended Sequence Number (ESN) support * NXP ARM64 SoC support * Addition of PowerPC64LE Architecture * ure - USB 3.0 Gigabit Ethernet Driver update * Stateless hardware offloads for VXLANs * Wireless updates * ZSTD Compression in ZFS Architectures * CheriBSD 2020 Q3 * FreeBSD/RISC-V Project Ports * Update to grub-bhyve * KDE on FreeBSD Documentation * DOCNG on FreeBSD Third-Party Projects * Potluck - Flavour & Image Repository for pot __________________________________________________________________ FreeBSD Team Reports Entries from the various official and semi-official teams, as found in the Administration Page. FreeBSD Foundation Contact: Deb Goodkin The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Funding comes from individual and corporate donations and is used to fund and manage software development projects, conferences and developer summits, and provide travel grants to FreeBSD contributors. The Foundation purchases and supports hardware to improve and maintain FreeBSD infrastructure and provides resources to improve security, quality assurance, and release engineering efforts; publishes marketing material to promote, educate, and advocate for the FreeBSD Project; facilitates collaboration between commercial vendors and FreeBSD developers; and finally, represents the FreeBSD Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity. Here are some highlights of what we did to help FreeBSD last quarter: COVID-19 Impact to the Foundation Like other organizations, we put policies in place for all of our staff members to work from home. We also put a temporary ban on travel for staff members. We are continuing our work supporting the community and Project, but some of our work and responses may be delayed because of changes in some of our priorities and the impact of limited childcare for a few of our staff members. Partnerships and Commercial User Support We help facilitate collaboration between commercial users and FreeBSD developers. We also meet with companies to discuss their needs and bring that information back to the Project. Not surprisingly, the stay at home orders, combined with our company ban on travel during Q3 made in-person meetings non-existent. However, the team was able to continue meeting with our partners and commercial users virtually. These meetings help us understand some of the applications where FreeBSD is used. We are currently scheduling Zoom company meetings for Q4, please reach out if you would like to schedule a meeting with us. Fundraising Efforts Last quarter we raised $192,874.43! Thank you to the individuals and organizations that stepped in, to help fund our efforts. We'd like to thank Arm for their large contribution last quarter, which helped bring our 2020 fundraising effort to $521k. We hope other organizations will follow their lead and give back to help us continue supporting FreeBSD. These are trying times, and we deeply appreciate every donation that has come in from $5 to $150,000. We're still here giving 110% to supporting FreeBSD! We are 100% funded by donations, and those funds go towards software development work to improve FreeBSD, FreeBSD advocacy around the world, keeping FreeBSD secure, continuous integration improvements, sponsoring BSD-related and computing conferences (even the virtual events!), legal support for the Project, and many other areas. Please consider making a donation to help us continue and increase our support for FreeBSD. We also have the Partnership Program, to provide more benefits for our larger commercial donors. Find out more information about the partnership program and share with your companies! OS Improvements A number of FreeBSD Foundation grant recipients started, continued working on, or completed projects during the third quarter. These include: * Ongoing WiFi and Linux KPI layer improvements. * Linuxulator application compatibility. * DRM / Graphics driver updates. * Zstd compression for OpenZFS. * Online RAID-Z expansion. * Modernized LLDB target support for FreeBSD. You can find more details about most of these projects in other quarterly reports. Staff members also worked on a number of larger projects, including: * Run-Time Dynamic Linker (rtld) and kernel ELF loader improvements. * Rewritten UNIX domain socket locking. * Build infrastructure. * Open system call path handling support for O_BENEATH, O_RESOLVE_BENEATH. * arm64 support. * Migration to a Git repository. Many of these projects also have detailed entries in other quarterly report entries. Staff members also put in significant effort in many ways other than larger, individual projects. These include assisting with code reviews, bug report triage, security report triage and advisory handling, addressing syzkaller reports, and ongoing maintenance and bug fixes in functional areas such as the tool chain, developer tools, virtual memory kernel subsystem, low-level x86 infrastructure, sockets and protocols, and others. University of Waterloo Co-op With the transition to working from home, the Foundation decided to again take on three University of Waterloo Co-op students for the Fall 2020 term (September to December). Tiger returns for a second term, joined by new students Yang and Zac. Projects for the term include more work on ELF Tool Chain, application of Capsicum to additional utilities, testing and integration of FreePBX and Asterisk VOIP software, pkgbase, and exploring containerization tooling. Continuous Integration and Quality Assurance The Foundation provides a full-time staff member and funds projects on improving continuous integration, automated testing, and overall quality assurance efforts for the FreeBSD project. During the third quarter of 2020, Foundation staff continued improving and monitoring the Project's CI infrastructure, and working with experts to fix the failing builds and the regressions found by tests. The setting up of dedicated VM host for running tests is completed. New feature developments and the CI staging environment is in progress. We are also working with other teams in the Project for their testing needs. For example, tests of non-x86 architectures now run periodically, and improve the CI of the embedded systems. We are also working with many external projects and companies to improve the CI between their products and FreeBSD. See the FreeBSD CI section of this report for completed work items and detailed information. Supporting FreeBSD Infrastructure The Foundation provides hardware and support to improve the FreeBSD infrastructure. Last quarter, we continued supporting FreeBSD hardware located around the world. We coordinated efforts between the new NYI Chicago facility and clusteradm to start working on getting the facility prepared for some of the new FreeBSD hardware we are planning on purchasing. NYI generously provides this for free to the Project. We also worked on connecting with the new owners of the Bridgewater site, where most of the FreeBSD infrastructure is located. Some of the purchases we made for the Project last quarter to support infrastructure includes: * Spamhaus spam filtering software to limit the amount of spam on the mailing lists. * 5 application servers to run tasks like bugzilla, wiki, website, cgi, Phabricator, host git, etc. * 1 server to replace the old pkg server and provide a lot more IOPS to avoid the slowdowns seen during peak times of the day where the disks just cannot keep up with the request volume. * 1 server for exp-runs to make them faster. * 1 server to build packages more frequently. FreeBSD Advocacy and Education A large part of our efforts are dedicated to advocating for the Project. This includes promoting work being done by others with FreeBSD; producing advocacy literature to teach people about FreeBSD and help make the path to starting using FreeBSD or contributing to the Project easier; and attending and getting other FreeBSD contributors to volunteer to run FreeBSD events, staff FreeBSD tables, and give FreeBSD presentations. The FreeBSD Foundation sponsors many conferences, events, and summits around the globe. These events can be BSD-related, open source, or technology events geared towards underrepresented groups. We support the FreeBSD-focused events to help provide a venue for sharing knowledge, to work together on projects, and to facilitate collaboration between developers and commercial users. This all helps provide a healthy ecosystem. We support the non-FreeBSD events to promote and raise awareness of FreeBSD, to increase the use of FreeBSD in different applications, and to recruit more contributors to the Project. As is the case for most of us in this industry, COVID-19 has put our in-person events on hold. In addition to attending virtual events, we are continually working on new training initiatives and updating our selection of how-to guides to facilitate getting more folks to try out FreeBSD. Check out some of the advocacy and education work we did last quarter: * Launched our FreeBSD Fridays series of 101 classes. Topics included an Introduction to FreeBSD, FreeBSD Installfest, Introduction to Security, Introduction to ZFS and more. Videos of the past sessions and a schedule of upcoming events can be found here. * Attended and presented at OSI's State of the Source conference. The event was held virtually, September 9-11, 2020. * Launched the redesign of the FreeBSD Foundation Website. * Announced the 20th Anniversary of the FreeBSD Foundation. * Participated as an Admin for Google Summer of Code 2020 * Continued to promote the FreeBSD Office Hours series including holding our own Foundation led office hours. Videos from the one hour sessions can be found on the Project's YouTube Channel. You can watch ours here. * Interviewed members of the outgoing FreeBSD Core Team to get their thoughts on their term. * Began working with the FreeBSD Vendor Summit planning committee on the November 2020 Vendor Summit. * Promoted the Foundation's 20th Anniversary and our work to support the FreeBSD Project in the It's FOSS Article. FreeBSD Foundation Celebrates 20 Years of Promoting and Supporting FreeBSD Project. * Authored a Beginners Guide to FreeBSD for Fosslife. * Committed to sponsoring All Things Open as a media Sponsor. * Committed to sponsoring the OpenZFS Developers Summit at the Bronze level. * Became an International RISC-V Member. * Committed to giving a FreeBSD talk at the nerdear.la conference on October 20th. Keep up to date with our latest work in our monthly newsletters. Netflix provided an update on how and why they use FreeBSD in our latest Contributor Case Study. We help educate the world about FreeBSD by publishing the professionally produced FreeBSD Journal. As we mentioned previously, the FreeBSD Journal is now a free publication. Find out more and access the latest issues at https://www.FreeBSDfoundation.org/journal/. You can find out more about events we attended and upcoming events at https://www.FreeBSDfoundation.org/news-and-events/. Legal/FreeBSD IP The Foundation owns the FreeBSD trademarks, and it is our responsibility to protect them. We also provide legal support for the core team to investigate questions that arise. We updated our Trademark Usage Terms and Conditions on July 1, 2020. Go to the FreeBSD Foundation's web site to find out how we support FreeBSD and how we can help you! ### Other We welcomed Andrew Wafaa and Kevin Bowling to our board of directors, to help govern the Foundation and guide us with our strategic direction. We have more information about our new board members on our website. __________________________________________________________________ FreeBSD Release Engineering Team Links FreeBSD 12.2-RELEASE schedule=20 URL: https://www.freebsd.org/releases/12.2R/schedule.html FreeBSD 12.2 test builds=20 URL: https://www.freebsd.org/where.html#helptest FreeBSD development snapshots=20 URL: https://download.freebsd.org/ftp/snapshots/ISO-IMAGES/ Contact: FreeBSD Release Engineering Team The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes and maintaining the respective branches, among other things. During the third quarter of 2020, the Release Engineering Team started work on the 12.2-RELEASE cycle, the third release from the stable/12 branch. As of this writing, two BETA builds have been released, with the expectation there will be a third BETA build currently remaining on the schedule. The 12.2-RELEASE cycle will continue throughout October, with two RC builds currently planned, and RC3 scheduled on an as-needed basis. The 12.2-RELEASE is so far scheduled for final release on October 27. In addition to the 12.2-RELEASE, Glen Barber of the Release Engineering Team finished work to the release build tools and scripts to prepare for the conversion from Subversion to Git for the 13.0-RELEASE cycle. There are no plans to merge these changes to stable branches at this time; as discussed within the Git working group, we feel such a change on a stable branch would be too intrusive to our user base as well as downstream FreeBSD consumers. Development snapshot builds for 13.0-CURRENT have recently been built from the Git tree within the project, and further snapshot builds for 12.x and 11.x will continue to be built from Subversion. Additionally throughout the quarter, several development snapshots builds were released for the head, stable/12, and stable/11 branches. Finally, the Release Engineering Team would like to thank Marius Strobl for his time serving on the team; he had recently stepped down from the Deputy RE Lead role due to constraints on his time. The Team welcomes Colin Percival, who has accepted fulfilling this role. Much of this work was sponsored by Rubicon Communications, LLC (netgate.com) and the FreeBSD Foundation. __________________________________________________________________ Cluster Administration Team Links Cluster Administration Team members=20 URL: https://www.freebsd.org/administration.html#t-clusteradm Contact: Cluster Administration Team The FreeBSD Cluster Administration Team consists of the people responsible for administering the machines that the Project relies on for its distributed work and communications to be synchronised. In this quarter, the team has worked on the following: * Work with the FreeBSD Foundation on hardware update for web services, mirror and package building servers. * Disable directory indexing on the package mirrors to resolve performance issues of the machine. + This was later relaxed to allow indexing of the parent directories but still disallow the large package directories. * Ongoing systems administration work: + Accounts management for committers. + Backups of critical infrastructure. + Keeping up with security updates in 3rd party software. Work in progress: * Setup Malaysia (KUL) mirror. * Setup Brazil (BRA) mirror. * Review the service jails and service administrators operation. * Infrastructure of building aarch64 and powerpc64 packages. + NVMe issues on PowerPC64 POWER9 blocking dual socket machine from being used as pkg builder. + Drive upgrade test for pkg builders (SSDs) courtesy of the FreeBSD Foundation. + Boot issues with Aarch64 reference machines. * New NYI.net sponsored colocation space in Chicago-land area. * Work with git working group for the git repository. * Searching for more providers that can fit the requirements for a generic mirrored layout or a tiny mirror. __________________________________________________________________ Continuous Integration Links FreeBSD Jenkins Instance=20 URL: https://ci.FreeBSD.org FreeBSD Hardware Testing Lab=20 URL: https://ci.FreeBSD.org/hwlab FreeBSD CI artifact archive=20 URL: https://artifact.ci.FreeBSD.org FreeBSD CI weekly report=20 URL: https://hackmd.io/@FreeBSD-CI FreeBSD Jenkins wiki=20 URL: https://wiki.freebsd.org/Jenkins Hosted CI wiki=20 URL: https://wiki.freebsd.org/HostedCI 3rd Party Software CI=20 URL: https://wiki.freebsd.org/3rdPartySoftwareCI Tickets related to freebsd-testing@=20 URL: https://preview.tinyurl.com/y9maauwg FreeBSD CI Repository=20 URL: https://github.com/freebsd/freebsd-ci Contact: Jenkins Admin Contact: Li-Wen Hsu Contact: freebsd-testing Mailing List Contact: IRC #freebsd-ci channel on EFNet The FreeBSD CI team maintains the continuous integration system of the FreeBSD project. The CI system firstly checks the committed changes can be successfully built, then performs various tests and analysis over the newly built results. The artifacts from those builds are archived in the artifact server for further testing and debugging needs. The CI team members examine the failing builds and unstable tests and work with the experts in that area to fix the codes or adjust test infrastructure. The details of these efforts are available in the weekly CI reports. During the third quarter of 2020, we continued working with the contributors and developers in the project to fulfill their testing needs and also keep collaborating with external projects and companies to improve their products and FreeBSD. Important changes: * All !x86 -test builds now trigger a new build on 22:00 UTC daily; this was not running very often because running all the tests in qemu takes lots of time. The work on improving the test execution speed and parallelism is in progress. The following is a list of the jobs affected: + Test build for FreeBSD HEAD on ARMv7. + Test build for FreeBSD HEAD on AArch64. + Test build for FreeBSD HEAD on MIPS64. + Test build for FreeBSD HEAD on PowerPC64. + Test build for FreeBSD HEAD on RISC-V64. * The build and test results will be sent to the dev-ci mailing list soon. Feedback and help with analysis is very appreciated! + A builder dedicated to run jobs using provisioned VMs is setup, this improves the stableness and reduces the execution time. + The result of FreeBSD-head-amd64-test_zfs is changed after OpenZFS importing; we encourage everyone to check and fix the failing and skipped test cases. New jobs added: * CI build for FreeBSD HEAD on PowerPC64LE. Work in progress: * Collecting and sorting CI tasks and ideas here. * Testing and merging pull requests in the the FreeBSD-ci repo. * Designing and implementing pre-commit CI building and testing, * Reduce the procedures of CI/test environment setting up for contributors and developers. * Setting up the CI stage environment and putting the experimental jobs on it. * Setting up public network access for the VM guest running tests. * Implementing automatic tests on bare metal hardware. * Adding drm ports building tests against -CURRENT. * Planning to run ztest and network stack tests. * Adding more external toolchain related jobs. * Improving the hardware lab to be more mature and adding more hardware. * Helping more 3rd software get CI on FreeBSD through a hosted CI solution. * Working with hosted CI providers to have better FreeBSD support. Please see freebsd-testing@ related tickets for more WIP information, and don't hesitate to join the effort! Sponsor: The FreeBSD Foundation __________________________________________________________________ Ports Collection Links About FreeBSD Ports=20 URL: https://www.FreeBSD.org/ports/ Contributing to Ports=20 URL: https://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing/= ports -contributing.html FreeBSD Ports Monitoring=20 URL: http://portsmon.freebsd.org/index.html Ports Management Team=20 URL: https://www.freebsd.org/portmgr/index.html Contact: Ren=C3=A9 Ladan Contact: FreeBSD Ports Management Team The Ports Management Team is responsible for overseeing the overall direction of the Ports Tree, building packages, and personnel matters. Below is what happened in the last quarter. We passed the landmark of 40,000 ports in the Ports Collection and are now around 40,400 ports. The last quarter saw 9335 commits to the HEAD branch and 481 commits to the 2020Q3 branch by respectively 167 and 63 committers. There are currently 2525 open problem reports of which 595 are unassigned. Compared to last quarter, this means a slight decrease in activity and also a slight increase in open PRs. During the last quarter we welcomed Rainer Hurling (rhurlin@) and said goodbye to Kevin Lo (kevlo@) and Grzegorz Blach (gblach@). The last three months saw new default versions for Perl (5.32), PostgreSQL (12) and PHP (7.4). Various packages also got updated: Firefox to 81.0.1, Chromium to 84.0.4147.135, Gnome to 3.36, Xorg to 1.20.9, Qt5 to 5.15.0, Emacs to 27.1, KDE Frameworks to 5.74.0 and pkg itself to 1.15.8. Never tired, antoine@ ran 30 exp-runs to test port version updates, on such diverse matters as: * Updating byacc in base to 20200330. * Check balancing of sed "y" command. * Use of brackets. * Removing the now redundant "port" argument from USES=3Dreadline. __________________________________________________________________ FreeBSD Office team - 3rd quarter 2020 report Links The FreeBSD Office project=20 URL: https://wiki.freebsd.org/Office Contact: FreeBSD Office team ML Contact: Dima Panov Contact: Li-Wen Hsu The FreeBSD Office team works on a number of office-related software suites and tools such as OpenOffice and LibreOffice. Work during this quarter focused on providing the latest stable release of LibreOffice suite and companion apps to all FreeBSD users. * Alongside with updating old stable branch to latest 6.4.x releases, current ports-tree now have a full-featured cutting-edge 7.0.1 bundle. * Conservative users can keep 6.4.x stable version by switching to use all-in-one editors/libreoffice6 port and even with i18n language pack (off by default). It will be kept updated at least till 7.1.0 version is released. We are looking for people to help the project. All unstable work with LibreOffice snapshots is staged in our WIP repository. The open bugs list contains all filed issues which need some attention. Patches, comments and objections are always welcome in the mailing list and bugzilla. __________________________________________________________________ FreeBSD Graphics Team status report Links Project GitHub page=20 URL: https://github.com/FreeBSDDesktop Contact: FreeBSD Graphics Team Contact: Niclas Zeising The FreeBSD X11/Graphics team maintains the lower levels of the FreeBSD graphics stack. This includes graphics drivers, graphics libraries such as the MESA OpenGL implementation, the X.org xserver with related libraries and applications, and Wayland with related libraries and applications. There have been several updates to the FreeBSD graphics stack and related libraries since the last report. Most notably, MESA related ports were changed to use the meson build system, instead of the autotools based one. This was needed since mesa upstream has deprecated and removed the autotools build system, and this paved the way for further mesa updates. While there was a need for a few minor corrections after the initial update, this update has been successful and made it possible to further update and improve the FreeBSD mesa port. There have also been several security fixes for xorg-server and libX11, so these ports have been updated to fix these issues. During the period, FreeBSD 12 was changed to improve the compatibility with input devices using udev/evdev and libinput. This change removes the need for local configuration and makes most mice, touchpads and keyboards work out of the box. This change will be in the upcoming FreeBSD 12.2 release. There have also been several updates to various libraries, both in the graphics and input stacks, and several userland drivers have been updated. Libraries such as libdrm and libevdev have been updated to include new FreeBSD support, developed by team members and added upstream. There has also been ongoing work to keep the various drm-kmod ports and packages up to date, mostly in response to changes in various FreeBSD versions. We have also continued our regularly scheduled bi-weekly meetings. People who are interested in helping out can find us on the x11@FreeBSD.org mailing list, or on our gitter chat. We are also available in #freebsd-xorg on EFNet. We also have a team area on GitHub where our work repositories can be found. __________________________________________________________________ Projects Projects that span multiple categories, from the kernel and userspace to the Ports Collection or external projects. FreeBSD on Microsoft HyperV and Azure Links Microsoft Azure article on FreeBSD wiki =20 URL: https://wiki.freebsd.org/MicrosoftAzure Microsoft HyperV article on FreeBSD wiki=20 URL: https://wiki.freebsd.org/HyperV Contact: FreeBSD Integration Services Team Contact: Wei Hu Contact: Li-Wen Hsu Li-Wen is working on the FreeBSD release code related to Azure for the -CURRENT, 12-STABLE and 11-STABLE branches. The work-in-progress is available here. The 11.4-RELEASE image on Azure Marketplace is published. We are testing the releng/12.2 branch and 12.2-RELEASE image will be published to Azure Marketplace soon after released. This project is sponsored by The FreeBSD Foundation, with resources provided by Microsoft. __________________________________________________________________ Building FreeBSD on non-FreeBSD hosts Links Wiki=20 URL: https://wiki.freebsd.org/BuildingOnNonFreeBSD Contact: Alex Richardson Until recently FreeBSD could only be built on a FreeBSD host. However, many popular free CI tools only allow building on Linux or macOS and therefore can not be used for building the FreeBSD base system. Furthermore, it is sometimes useful to cross-build FreeBSD for a remote machine or an emulator even if the build machine is not running FreeBSD. The goal of this project is to allow building the base system on Linux and macOS hosts. I started this project in 2017 to allow building CheriBSD on the Linux servers and desktops that many of us working on the CHERI project use. The first few patches were upstreamed in 2018 (see the 2018q3 report) and I merged the full set of patches to CheriBSD shortly after. Over the past two years I have slowly been upstreaming the remaining patches and finally committed the last required change in time for this report. As of September 2020 it should be possible to use the buildworld and buildkernel make targets to build a fully-functional FreeBSD installation on macOS and Linux hosts. We use this in our continuous integration system to build and test CheriBSD disk images for multiple architectures. I have also committed a GitHub Actions configuration upstream that takes approximately 10 minutes to build an amd64 kernel. This will ensure that changes that break crossbuilding from Linux/macOS can be detected easily. Upstreaming the crossbuilding changes has resulted in various build system cleanups. For example, we now no longer need to use lorder.sh when building libraries which speeds up the linking step a bit. The portability and bootstrapping changes should also make it easier to upgrade from older versions since we no longer rely on host headers in /usr/include matching those of the target system (e.g. when bootstrapping localedef, etc.). While this support for building on Linux and macOS should still be considered experimental, it should work in many cases. If you would like to give it a try, the following command line should successfully build an amd64 world on Linux and macOS systems that have packages for LLVM 10 (or newer) installed: MAKEOBJDIRPREFIX=3D/somewhere ./tools/build/make.py TARGET=3Damd64 TARGET_ARCH=3Damd64 buildworld Buil= ds must be performed using the ./tools/build/make.py wrapper script since most Linux and macOS systems do not ship an appropriate version of bmake. Please let me know if you encounter any issues. Sponsor: DARPA __________________________________________________________________ Git Migration Working Group Links Git conversion tooling repo=20 URL: https://github.com/freebsd/git_conv FreeBSD-git mailing list=20 URL: https://lists.freebsd.org/mailman/listinfo/freebsd-git Beta doc git repo=20 URL: https://cgit-beta.FreeBSD.org/doc Beta ports git repo=20 URL: https://cgit-beta.FreeBSD.org/ports Beta src git repo=20 URL: https://cgit-beta.FreeBSD.org/src Contact: Ed Maste Contact: Warner Losh Contact: Ulrich Sp=C3=B6rlein Work continues on FreeBSD's migration from Subversion to Git. Ulrich has addressed all known issues with svn2git and has been able to work around the inconsistent metadata and forced commit issues in the Subversion history. We still have additional documentation to write, and need to finish installing commit hooks (e.g. restricting branch creation, or ensuring appropriate data exists on cherry-pick commits). We expect to open the beta repository to test commits before the end of October. This is to allow testing of the commit hooks, and to allow developers to test access and become familiar with git operation. Commits in this repository will be deleted and the repository will be recreated at least once prior to the final migration. Those with an interest in the migration to Git are encouraged to subscribe to the FreeBSD-git mailing list and test out the beta src, ports, and/or doc repositories. You are also welcome check out the wiki, issues, README and other documentation at the Git conversion tooling repo. We currently expect to transition the src and doc repositories in mid-November. Additional investigation and experimentation with the ports repository is still underway. Sponsor: The FreeBSD Foundation (in part) __________________________________________________________________ Linux compatibility layer update Contact: Edward Tomasz Napierala Contact: Mark Johnston Earlier Linuxulator work focused on code cleanups and improving diagnostic tools. Work has now shifted from cleanups to fixing actual applications. Current status is being tracked at Linux app status Wiki page. Initial focus was on applications that don't involve X11, mostly because they tend to be easier to test and debug, and the bug fixes are not application-specific. Foundation-sponsored work during this quarter included implementing a devfs(5) workaround to fix gettynam(3) inside jail/chroot, and workaround for the missing splice(2) syscall, which caused problems for grep and autotools. The Linux version reported to userspace was bumped to 3.10.0, which matches the kernel shipped with RHEL 7 and is neccessary for IBM's DB2 database installation to succeed. The BLKPBSZGET ioctl neccessary for Oracle database is supported now. There is now support for kcov(4), neccessary for syzcaller; as well as a number of fixes for issues reported by syzcaller, such as futex lock leaks. There were also more cleanups, including moving some Linuxulator-specific functionality related to error handling off from the syscall's fast code paths. The sysutils/debootstrap port, which provides an easy way to create Debian or Ubuntu jail, was updated to version 1.0.123. Finally there were some improvements to the documentation. Most of those changes have been merged to FreeBSD 12-STABLE, in order to ship with 12.2-RELEASE. There is increased involvement from other developers; this includes termios performance fixes, improved memfd support, implementing CLOCK_MONOTONIC_RAW required for Steam, madvise improvements, new compat.linux.use_emul_path sysctl. There is also ongoing work on tracking down the causes of failures related to Steam and WebKit, with fixes being first implemented in linuxulator-steam-utils. Sponsor: The FreeBSD Foundation __________________________________________________________________ LLDB Debugger Improvements Links Moritz Systems Project Description=20 URL: https://www.moritz.systems/blog/lldb-debugger-improvements-for-fre= ebsd/ Git Repository=20 URL: https://github.com/moritz-systems/llvm-project Contact: Kamil Rytarowski Contact: Michal G=C3=B3rny FreeBSD includes LLDB, the debugger in the LLVM family, in the base system. At present it has some limitations in comparison with the GNU GDB debugger, and does not yet provide a complete replacement. It relies on an obsolete plugin model in LLDB that causes growing technical debt. This project aims to bring LLDB closer to a fully featured replacement for GDB, and therefore for FreeBSD to feature a modern debugger for software developers. The legacy monolithic target supports the executed application being debugged in the same process space as the debugger. The modern LLDB plugin approach, used on other supported targets, executes the target process under a separate lldb-server process. This improves reliability and simplifies the process / thread model in LLDB itself. In addition, remote and local debugging will both be performed using the same approach. After the migration to the new process model is complete, the project will include reviewing the results of LLDB's test suite and fixing tests as time permits. The work is expected to be complete in 2020. The project schedule is divided into three milestones, each taking approximately one month: 1. Introduce new FreeBSD Remote Process Plugin for x86_64 with basic support and upstream to LLVM. 2. Ensure and add the mandated features in the project (process launch, process attach (pid), process attach (name), userland core files, breakpoints, watchpoints, threads, remote debugging) for FreeBSD/amd64 and FreeBSD/i386. 3. Iterate over the LLDB tests. Detect, and as time permits, fix bugs. Ensure bug reports for each non-fixed and known problem. Add missing man pages and update the FreeBSD Handbook. We are nearing the completion of the first milestone. The new plugin is getting into shape, and it can already run simple single-threaded programs. The supported features include single-stepping, breakpoints, memory and register I/O on amd64. Both plugins are supported simultaneously. The new plugin is used if FREEBSD_REMOTE_PLUGIN environment variable is set to any value, or if lldb-server is spawned directly. Otherwise, the old plugin is used for compatibility. Once the new plugin matures, we are planning to enable it unconditionally on the architectures that it is ported to. Sponsor: The FreeBSD Foundation __________________________________________________________________ Lua usage in FreeBSD Contact: Ed Maste Contact: Kyle Evans Contact: Ryan Moeller During this quarter, flua (FreeBSD Lua) was taught where to find base .lua modules in order to support require of .lua modules to be provided by the base system. flua also gained support for require of binary modules. A review for libjail bindings has also been submitted, pending review. libjail is an essential component if one wants to be able to write jail management utilities in flua. People interested in working with Lua in FreeBSD are welcome to get in contact to discuss other project ideas. To name a couple of potential projects, some interesting modules that have not been started but could prove useful (listed in no particular order): * libcrypt * libexpat * libnv * libxo There is also a small list of scripts that would do well with a port to flua: * certctl(8) __________________________________________________________________ NFS over TLS implementation Contact: Rick Macklem In an effort to improve NFS security, an internet draft which I expect will become an RFC soon specifies the use of TLS 1.3 to encrypt all data traffic on a Sun RPC connection used for NFS. Although NFS has been able to use sec=3Dkrb5p to encrypt data on the wire, this requires a Kerberos environment and, as such, has not been widely adopted. It also required that encryption/decryption be done in software, since only the RPC message NFS arguments are encrypted. Since Kernel TLS is capable of using hardware assist to improve performance and does not require Kerberos, NFS over TLS may be more widely adopted, once implementations are available. The coding for this project has now been completed. All required changes to the NFS and kernel RPC code have been committed to -CURRENT. The daemons are now believed to be complete, but will remain in base/projects/nfs-over-tls until -CURRENT has an OpenSSL library with the kernel TLS support incorporated in it. If this does not happen for FreeBSD-13, hopefully the patched OpenSSL and the daemons can become ports. To support clients such as laptops, the daemons that perform the TLS handshake may optionally handle client X.509 certificates from a site local CA. There are now exports(5) options to require client(s) to provide a valid X.509 certificate. While setting up system(s) for testing is still a little awkward, the documentation is now available for those who want to help with testing. The main limitation in the current implementation is that it uses TLS1.2 and not TLS1.3. This should change once the KERN_TLS rx patch includes TLS1.3 support. Third party testing would be appreciated. __________________________________________________________________ syzkaller on FreeBSD Contact: Mark Johnston See the syzkaller entry in the 2019q1 quarterly report for an introduction to syzkaller. syzkaller, especially the public syzbot instance, continues to find bugs in the FreeBSD kernel. A number of these bugs have been fixed in subsystems such as the VFS name cache, the TCP and SCTP stacks, pf(4), the unix domain socket implementation, and the Linuxulator. The FreeBSD Foundation sponsored some work to enable cross-OS fuzzing. This makes it possible to fuzz the Linuxulator using syzkaller's Linux target. This effort quickly found several bugs; once the support is committed upstream we will hopefully be able to leverage syzbot to gain continuous testing of the Linux system call interface in addition to the native and 32-bit compatibility interfaces. Some work was also done to enable running syzkaller in a FreeBSD jail, with the eventual aim of making it easy to distribute binary images containing everything required to immediately start running syzkaller on a new host. Currently a number of setup steps are required, making deployment somewhat painful. Sponsor: The FreeBSD Foundation __________________________________________________________________ Kernel Updates to kernel subsystems/features, driver support, filesystems, and more. DRM Drivers Update Links drm-kmod=20 URL: https://github.com/freebsd/drm-kmod/ Contact: Emmanuel Vadot The drm drivers for FreeBSD 13-CURRENT have been updated to match Linux 5.4.62 Then graphics/drm-current-kmod have been updated to follow this LTS release of Linux. For now graphics/drm-devel-kmod is also tracking this release but will be updated to a later revision of Linux drm drivers in the near future. A lot of linuxkpi code was removed from the ports or replaced with a BSD licenced implementation. Sponsor: The FreeBSD Foundation __________________________________________________________________ DTS Update Contact: Emmanuel Vadot DTS files (Device Tree Sources) were updated to be on par with Linux 5.8 for HEAD and 5.6 for the 12-STABLE branch. __________________________________________________________________ DesignWare Ethernet adapter driver improvements Links WIP branch=20 URL: https://github.com/gonzoua/freebsd/tree/rk_eth Contact: Oleksandr Tymoshenko DesignWare Ethernet adapter IP is used in Rockchip and Allwinner SoCs. The driver was updated with following fixes: * Initialize clocks instead of relying on u-boot to do the right thing. * Sense media type and adjust controller configuration accordingly. * Add support for RMII PHY mode. Yet uncommitted changes include performance optimisation by adding support for multi-segment mbuf transmission. The next step is to try to get more performance boost by using interrupt coalescence. __________________________________________________________________ Google Summer of Code'20 Project - eBPF XDP Hooks Links Github diff link=20 URL: https://github.com/Ankurk99/freebsd/tree/ebpf-import Project wiki =20 URL: https://wiki.freebsd.org/SummerOfCode2020Projects/eBPFXDPHooksl Contact: Ankur Kothiwal The eBPF eXpress Data Path (XDP) allows eBPF programs to be run to filter received packets as early as possible, avoiding unnecessary processing overhead before the filter is run. The goal of this project is to extend an existing FreeBSD network driver (a virtual NIC like a VirtIO if_vtnet) to be able to call into an eBPF program when processing a newly received packet. In short, with XDP the driver must PASS (accept and process normally), DROP, TX or REDIRECT the packet as specified by the program. eBPF helper functions and maps for aiding in packet filtering will also be implemented. Implemented: * Register a eBPF probe when an interface is registered with pfil. * Activating eBPF probe. * Create hooks and link them to the pfil head when the eBPF XDP probe is activated and successfully list the XDP probes. * Create a xdp_rx function which will pass the received packets to the eBPF program where the packets can be further processed. This function will return XDP actions: DROP and PASS. * Register the xdp hook and link it to the pfil head. * Write an eBPF program to process (currently drop and pass) ICMP traffic - This is to test that the hook is working properly. * Write a loader function to load the ICMP filter program to the kernel. Future Work: * Currently we can only attach the XDP hook to PASS and DROP the packets - The work on detaching the hook is left. * The XDP action to "TX" and "REDIRECT" the packets. Final Deliverables: * Implemented XDP hook to pass and drop packets. * Created a loader program to attach the eBPF program to the kernel. * A test program to DROP ICMP filter. This code was done under the Google Summer of Code 2020 under the guidance of Ryan Stone (rstone@). The eBPF implementation for FreeBSD is still a work in progress and FreeBSD doesn't support eBPF yet. The basic implementation for eBPF was a GSoC'18 project, and is still under development. This project is based on that implementation so the XDP implementation for FreeBSD can only be merged into the FreeBSD source code once it supports eBPF. Currently this code is a work in progress and is merged to Ryan Stone's branch with support for the eBPF implementation. Sponsor: Google Summer of Code __________________________________________________________________ ENA FreeBSD Driver Update Links ENA README=20 URL: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/ena/R= EADME Contact: Michal Krawczyk Contact: Artur Rojek Contact: Marcin Wojtas ENA (Elastic Network Adapter) is the smart NIC available in the virtualized environment of Amazon Web Services (AWS). The ENA driver supports multiple transmit and receive queues and can handle up to 100 Gb/s of network traffic, depending on the instance type on which it is used. Completed since the last update: * Fix ENA compilation in case it is integrated into the kernel binary. * MFC of the ENA v2.2.0 driver to the FreeBSD 12.2. Work in progress: * Add feature that allows reading extra ENI (Elastic Network Interface) metrics about exceeding BW/pps limits. * Introduce full kernel RSS API support. * Allow reconfiguration of the RSS indirection table and hash key. * Evaluation and prototyping of the driver port to the iflib framework. Sponsor: Amazon.com Inc __________________________________________________________________ IPSec Extended Sequence Number (ESN) support Contact: Grzegorz Jaszczyk Contact: Patryk Duda Contact: Marcin Wojtas Extended Sequence Number (ESN) is IPSec extension defined in RFC4303 Section 2.2.1. It makes possible to implement high-speed IPSec implementations where standard, 32-bit sequence number is not sufficient. A key feature of the ESN is that only low order 32 bits of sequence number are transmitted over the wire. High-order 32 bits are maintained by sender and receiver. Additionally high-order bits are included in the computation of Integrity Check Value (ICV) field. Extended Sequence Number support contains following: * Modification of existing anti-replay algorithm to fulfil ESN requirements. * Trigger soft lifetime expiration at 80% of UINT32_MAX when ESN is disabled. * Implement support for including ESN into ICV in cryptosoft engine in both encrypt and authenticate mode (eg. AES-CBC and SHA256 HMAC) and combined mode (eg. AES-GCM). * Implement support for including ESN into ICV in AES-NI engine in both encrypt and authenticate mode and combined mode. Completed since the last update: * Adjust implementation of crypto part to the reworked Open Crypto Framework. * Move the core ESN implementation from the crypto drivers to netipsec layer. * Make use of the newly introduced crp_aad mechanism for combined modes. * Introduce minor fixes and improvements. TODO: * Complete review process in Phabricator and merge patches in the tree. Sponsor: Stormshield __________________________________________________________________ NXP ARM64 SoC support Contact: Marcin Wojtas Contact: Artur Rojek Contact: Dawid Gorecki The Semihalf team initiated working on FreeBSD support for the NXP LS1046A SoC LS1046A are quad-core 64-bit ARMv8 Cortex-A72 processors with integrated packet processing acceleration and high speed peripherals including 10 Gb Ethernet, PCIe 3.0, SATA 3.0 and USB 3.0 for a wide range of networking, storage, security and industrial applications. Completed since the last update: * Upstreaming of the QorIQ SDHCI driver (r365054). With above the current Semihalf upstreaming activity is complete. The major out-of-tree supported components: * DPAA network controller support. * QSPI controller support. They work on 11.2-RELEASE, but still require significant effort to adopt to FreeBSD-CURRENT. Sponsor: Alstom Group __________________________________________________________________ Addition of PowerPC64LE Architecture Links Early notes=20 URL: https://lists.freebsd.org/pipermail/freebsd-ppc/2020-August/012043= =2Ehtml Announcement=20 URL: https://lists.freebsd.org/pipermail/freebsd-ppc/2020-September/012= 098.h tml Contact: Brandon Bergren As of r366063, experimental support for little-endian PowerPC64 (PowerPC64LE) is available in -CURRENT for POWER8 and POWER9 machines. In 2010, when FreeBSD was ported to PowerPC64, the average user would have been using a G5 PowerMac, a purely big-endian machine. While, at the time, a 32-bit PowerPC machine could run in little-endian, as well as POWER6 and POWER7, in practice, the complexities involved in managing it at the kernel level and lack of firmware support made it infeasible to support. When IBM designed POWER8, one main focus was to improve little-endian support, and bring it up to parity with big-endian. This improved support makes it practical to support a little-endian operating environment on what is traditionally a primarily big-endian platform. In 2020, with POWER9 being affordable for many users thanks to the Raptor Blackbird, semi-easy access to surplus POWER8 hardware, IBM having a major future focus on POWER little-endian, and the decay of big-endian support in modern video cards and graphical environments, there is demand for a little-endian version of FreeBSD on POWER. With FreeBSD/PowerPC64's transition in 2019 to the ELFv2 ABI as part of the 2019q4 PowerPC on Clang effort, the last major barrier to a little-endian port was eliminated. Since nobody else was working on it, and I had the skillset required to do the port, I decided to experiment one weekend with a little-endian kernel to see how difficult it would be to port. It turned out to be a lot more trivial than I was expecting. Three days later I had console support in qemu, and after another week of debugging, I had it fully up and running on hardware. FreeBSD PowerPC64LE is now an experimental MACHINE_ARCH in base, and is continuing to evolve at a rapid pace. Big-endian PowerPC64 is still the preferred platform for the foreseeable future, and will not be deprecated. Sponsor: Tag1 Consulting, Inc. __________________________________________________________________ ure - USB 3.0 Gigabit Ethernet Driver update Links svn commit: r365648=20 URL: https://svnweb.freebsd.org/changeset/base/365648 FreeBSD-SA-20:27.ure=20 URL: https://www.freebsd.org/security/advisories/FreeBSD-SA-20:27.ure.a= sc D25809 major update to if_ure=20 URL: https://reviews.freebsd.org/D25809 Contact: John-Mark Gurney The ure is a driver for handling the RealTek ethernet adapters, including the RTL8153 USB 3.0 Gigabit ethernet adapters. It is used in many ethernet dongles and docking stations. Previous to this update, the driver was limited in speed. In my testing, I was only able to get ~91Mbps. This limit was due to one packet per USB transfer. USB has a limit of 8000 transfers per second (1500 bytes/pkt * 8000 pkts/sec * 8bits/byte =3D=3D 96 Mbps). This was acceptable for fast ethernet (RTL8152, 100Mbps), but with the additional support for Gigabit ethernet, it became a bottleneck. The updates add sending and receiving multiple packets in a single USB transfer, VLAN hardware tagging, and enable TCP and UDP checksum offloading. This increased the speed on gigabit ethernet to ~940 Mbps. In doing this work, a security vulnerability was discovered in the driver. Due to improper setting of a device register, on some devices, it caused packets to be fragmented when they shouldn't be and the driver was unable to handle them correctly. This allowed an attacker, who could generate large frames (say, ping packets, or large TCP transfers), to inject arbitrary packets into the network stack. This could allow the attacker to spoof traffic from other machines, and bypass VLAN protections. See the SA for more information. As part of this work, a script was created to run tests to validate that basic functionality of the driver (w/o options) work properly, and then iterate over each option to make sure that they function properly. This will be released at some point in the future. If you're interested in helping out, or testing it, let me know. __________________________________________________________________ Stateless hardware offloads for VXLANs Links r365867=20 URL: https://svnweb.freebsd.org/base?view=3Drevision&revision=3Dr365867 r365868=20 URL: https://svnweb.freebsd.org/base?view=3Drevision&revision=3Dr365868 r365869=20 URL: https://svnweb.freebsd.org/base?view=3Drevision&revision=3Dr365869 r365870=20 URL: https://svnweb.freebsd.org/base?view=3Drevision&revision=3Dr365870 r365871=20 URL: https://svnweb.freebsd.org/base?view=3Drevision&revision=3Dr365871 RFC6935=20 URL: https://tools.ietf.org/html/rfc6935 Contact: Navdeep Parhar VXLAN (Virtual eXtensible LAN) is a tunneling protocol in which Layer 2 traffic for a virtual LAN is encapsulated in UDP and transferred over Layer 3 networks between VTEPs (VXLAN Tunnel End Points). Traffic on the wire has two sets of networking headers: the headers for the encapsulation and the headers of the traffic being encapsulated. VXLANs are supported by if_vxlan(4) on FreeBSD. Modern NICs commonly support header checksum insertion and verification, TSO (TCP Segmentation Offload) on transmit, and RSS for load distribution on receive. But the default is to operate on the outermost headers. Some NICs can operate on the inner encapsulated frames as well. The commits listed above allow if_vxlan(4) to take advantage of such NICs. r365867 and r365868 add new mbuf checksum flags and ifnet capabilities. r365870 implements the kernel parts of the new capabilities and updates if_vxlan(4) to make use of them. r365871 implements driver support for the new capabilities in cxgbe(4). VXLAN and other tunneling protocols that use UDP explicitly allow zero checksum in the outer UDP header, even with IPv6. r365869 adds support for configuring one UDP/IPv6 port where zero checksums are allowed. This work was sponsored by Chelsio Communications and was implemented and tested using T6 (Terminator 6) NICs supported by cxgbe(4). It is available in 13.0-CURRENT (head) right now and will be available in 12-STABLE in the future. VXLANs can be created as usual and will automatically have checksum and TSO capabilities if the underlying physical interface supports VXLAN stateless offloads. Use ifconfig to list, disable, and enable checksum capabilities on the VXLAN interface. Use https://bugs.freebsd.org/bugzilla/ to report bugs. Future work: * Direct call into a vxlan input routine from the driver's receive routine. * LRO support in if_vxlan(4). * GENEVE support. Sponsor: Chelsio Communications __________________________________________________________________ Wireless updates Links The freebsd-wireless mailing list=20 URL: https://lists.freebsd.org/mailman/listinfo/freebsd-wireless athp github repository=20 URL: https://github.com/erikarn/athp Contact: Adrian Chadd Contact: Bjoern A. Zeeb The following works happened in FreeBSD HEAD (some already in Q2) and were merged for 12.2-BETA2 and include net80211 and driver updates for better 11n and upcoming 11ac support. In more detail, this includes an ath(4) update, some run(4) 11n support, 11n for otus(4), A-MPDU, A-MSDU, A-MPDU+A-MSDU and Fast frames options, scanning fixes, enhanced PRIV checks for jails, restored parent device name printing, improvements for upcoming VHT support, lots of under-the-hood infrastructure improvements, new device IDs, and debug tools updates. If you have a chance please test before the release. Atheros 11ac driver athp In the last three months the athp(4) port of the ath10k driver has progressed well. Adrian reports the following important changes: * Per-node transmit buffering was implemented, required for correct hostap and QCA6174 behaviour. * Issues with ignoring sending some management frames got fixed; null-data frames were being filtered out and this caused undesirable hostap behaviour. * Transmit path refactoring reduced code duplication. * A fix on firmware start / VAP running tracking no longer stops the first VAP from coming active after VAP creation / ifconfig up. * Correcting hostap mode PHY configuration now allows non-VHT stations to associate and correctly exchange data with a VHT AP. * Addition of a crypto key configuration cache in the driver ensures the ieee80211_key details are available after the key is deleted; net80211 would reuse or free the state before the driver task would finish the firmware command. Newer Intel Wireless device support Initial work was done to integrate net80211 support in the LinuxKPI compat layer to get the wireless parts going. In addition, upstreaming code changes and working through problems and review started on two sides. One was trying to get mostly compile time changes upstreamed to the iwlwifi driver. The other is sorting out conflicting LinuxKPI changes to not break the DRM graphics drivers. Bjoern hopes that with some of that sorted out, he can soon go back to focus on the wireless parts and produce a new snapshot. rtw88 and brcmfmac As the Intel driver port and LinuxKPI advance, both the rtw88, and to a lower degree the brcmfmac, ports benefit from that. Bjoern lately also got a brcmfmac PCIe card and started to port support for that. This for the moment remains a free-time project. Work by Bjoern was sponsored by: Rubicon Communications, LLC (d/b/a "Netgate") and The FreeBSD Foundation __________________________________________________________________ ZSTD Compression in ZFS Contact: Allan Jude Zstandard (ZSTD) is a modern high-performance compression algorithm designed to provide the compression ratios of gzip while offering much better performance. ZSTD has been adopted in FreeBSD for a number of other uses, including compressing kernel crash dumps, as a replacement for gzip or bzip for compressing log files, and for future versions of pkg(8). This effort to complete the integration of ZSTD into ZFS is funded by the FreeBSD Foundation. During the third quarter the integrating of ZSTD into OpenZFS was completed in the upstream OpenZFS repository, and the new OpenZFS 2.0 codebase was imported into 13-CURRENT. Completed milestones in this project: * Importing ZSTD 1.4.5 into OpenZFS, using the recent upstream zstd features that make it easier to embed zstd in other projects. * Changing the way compression levels are tracked and inherited. * Save and restore the compression level via an embedded block header. * Also store the version of zstd used in the embedded block header, for future-proofing. The checksum of a block may not match if zstd is upgraded, since it may compress the block more. * Add tests to ensure zstd compression and metadata survive ZFS replication. * Resolve possible negative interactions with L2ARC and ZFS Native Encryption. * Fix bug with L2ARC if the Compressed ARC feature is disabled. * Improve the ZFS feature activation code, so that zstd cannot create pools that will panic older versions of ZFS. With these changes, upgraded pools can compress data with zstd or zstd-fast, across a wide range of different compression levels. This will allow the storage administrator to select the performance-vs-compression tradeoff that best suits their needs. Tasks remaining to be completed: * Add a section to the FreeBSD Handbook ZFS chapter about zstd * Create more documentation around selecting a suitable compression level * Finish support for ZSTD in the FreeBSD boot loader (Warner Losh imp@freebsd.org) Sponsor: The FreeBSD Foundation __________________________________________________________________ Architectures Updating platform-specific features and bringing in support for new hardware platforms. CheriBSD 2020 Q3 Links Contact: Alex Richardson Contact: Andrew Turner Contact: Brooks Davis Contact: Edward Tomasz Napierala Contact: George Neville-Neil Contact: Jessica Clarke Contact: John Baldwin Contact: Robert Watson Contact: Ruslan Bukin CheriBSD extends FreeBSD to implement memory protection and software compartmentalization features supported by the CHERI instruction-set extensions. There are three architectural implementations of the CHERI protection model: CHERI-MIPS, CHERI-RISC-V, and Arm's forthcoming experimental Morello processor (due late 2021). CheriBSD is a research operating system with a stable baseline implementation into which various new research features have been, or are currently being, merged: * Arm Morello - We are preparing to open source our adaptation of CheriBSD to Arm's Morello architecture. The Morello branch is being updated to the most recent CheriBSD baseline, and patches are in review for upstreaming to our open-source repository. CheriBSD currently boots and runs statically linked CheriABI binaries on the Morello simulator, and dynamic linking support is in progress, with OS and toolchain bugs being worked on. We aim to make a first CheriBSD/Morello snapshot available alongside other open-source Morello software in mid-October 2020, however, our target for a more mature and usable implementation is December 2020. * Kernel spatial memory safety (pure-capability kernel) - The current CheriBSD kernel is a hybrid C program where only pointers to userspace are CHERI capabilities. This ensures that the kernel follows the intent of the application runtime and cannot be used to defeat bounds on application pointers. We have developed and will soon merge a pure-capability kernel where all pointers in the kernel are appropriately bounded capabilities. This vastly reduces the opportunity for buffer overflows. This spatial memory safety lays the groundwork for future work such as device driver compartmentalization and kernel temporal safety. * Userspace heap temporal memory safety (Cornucopia) - CHERI capabilities provide the necessary features to enable robust and efficient revocation of freed pointers. With Cornucopia we have implemented a light-weight revocation framework providing protection from use-after-reallocation bugs with an average cost below 2%. We aim to bring these overheads down further over the next year and merge this functionality into the mainline CheriBSD. * We have been working on updating the arm64 bhyve from Politehnica University of Bucharest to have it committed to FreeBSD. We have been upstreaming initial changes to help support this. * Baseline FreeBSD improvements - We are upstreaming (to FreeBSD) various bug fixes and tweaks for PCIe support, and support for the System MMU (SMMU) that will be present on the N1SDP and Morello SoCs. We have upstreamed support for cross-building FreeBSD from macOS and Linux (with some limitations; see separate entry on crossbuilding). We have also fixed implementation bugs in the RISC-V ABI. CHERI Documentation and Exercises * We have released [Capability Hardware Enhanced RISC Instructions: CHERI Instruction-Set Architecture (Version 8)](https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-951.pdf). Notable changes include promotion of CHERI-RISC-V to non-experimental and discussion of Arm's Morello prototype. * We have developed a set of [Adversarial CHERI Exercises and Missions](https://ctsrd-cheri.github.io/cheri-exercises) to introduce security researchers to CHERI protections. __________________________________________________________________ FreeBSD/RISC-V Project Links Wiki=20 URL: https://wiki.freebsd.org/riscv Contact: Mitchell Horne Contact: freebsd-riscv Mailing List Contact: IRC #freebsd-riscv on freenode The FreeBSD/RISC-V project is providing support for running FreeBSD on the RISC-V Instruction Set Architecture. This quarter saw several important bug fixes. A number of hangs in the system were identified and addressed, and a bug in QEMU's implementation of the Platform Level Interrupt Controller was fixed. This fix is included in the new devel/qemu50 and devel/qemu-devel ports. The end result of these fixes is that the test suite can now be reliably run to completion in QEMU. The entire run takes several hours, so CI has been configured to run the job once a day. There is active effort into reducing the time it takes to run the entire test suite. A new u-boot port was created: sysutils/u-boot-qemu-riscv64. This variant can be used as a secondary bootloader alongside OpenSBI to load and launch FreeBSD's loader(8) from an EFI System Partition. Next quarter will likely bring further fixes to address some of the failing test cases. __________________________________________________________________ Ports Changes affecting the Ports Collection, whether sweeping changes that touch most of the tree, or individual ports themselves. Update to grub-bhyve Links grub-bhyve Git Repository=20 URL: https://gitlab.com/ctuffli/grub Contact: Chuck Tuffli bhyve is the hypervisor included in FreeBSD and other operating systems used to run virtual machines. When not using a boot ROM (i.e. UEFI), the user must load the guest operating system for bhyve. For non-FreeBSD guests, the loader is a version of GNU GRUB (a.k.a the GNU GRand Unified Bootloader) modified to interface with bhyve. This work is an effort to both update the base GRUB code to the latest version as well as improve the usability on FreeBSD. The current grub-bhyve is based on an older version of GRUB (circa 2015) and thus is missing more recent additions such as XFS file system and syslinux support. With the update, installing CentOS, for example, now does not require the extra step of changing the default file system to something other than XFS. Internally, the code has been restructured to be its own "platform" which should make it easier to keep in sync with upstream development. The major improvement is the ability to automatically find and load the GRUB configuration file from the guest disk image. With this change, it is not necessary to create a device map file or specify which Linux kernel or initrd image to use. More importantly, if the guest image updates its GRUB configuration, for example after updating the kernel, no changes are needed when invoking grub-bhyve. Note, this feature requires a new "disk" option: # grub-bhyve --disk=3D/zroot/vms/u18-mini/disk0.img --vm=3Du18-mini The automatic configuration file detection works with both GRUB configuration files (e.g. CentOS, Ubuntu) as well as syslinux configuration (e.g. Alpine). For the adventurous, there is experimental support for Fedora's BootLoaderSpec (a.k.a. blscfg) on the blscfg branch of the grub-bhyve Git repository. The code has been tested on a few Linux variants, but it would benefit from wider testing (and bug reports!). The new version does not have a Port but is easily built on FreeBSD. After cloning / downloading the source, run: $ PYTHON=3Dpython3.7 ./bootstrap $ MAKE=3Dgmake ./configure --with-platform=3Dbhyve $ gmake The resulting binary, grub-bhyve, will be in the grub-core/ directory. If you have success or troubles with it, please let me know. __________________________________________________________________ KDE on FreeBSD Links KDE FreeBSD =20 URL: https://freebsd.kde.org/ KDE Community FreeBSD=20 URL: https://community.kde.org/FreeBSD Contact: Adriaan de Groot The KDE on FreeBSD project aims to package all of the software produced by the KDE Community for the FreeBSD ports tree. The software includes a full desktop environment called KDE Plasma, an IDE with the name KDevelop, a PIM suite known as Kontact and hundreds of other applications that can be used on any FreeBSD machine. With the continuation of the ever-so-peculiar era of almost-only-online, the KDE community has shifted gears and also gone for online events. The yearly conference, Akademy, was conducted online over video calls. Meanwhile, software continues to be released, so this quarter the kde@ team: * Put the beta of the next version of KDE Plasma, scheduled for official release in October 2020, into the Area51 development tree. Area51 is a fork of the FreeBSD ports tree where new development for KDE ports happens. * The monthly regular updates to the KDE Plasma desktop landed on-time and safely. * With three months in a quarter, there were also three releases of KDE Frameworks 5, including a new framework for handling DAV jobs. * The June applications update and its .1 release landed a bit late, but brings with it the usual raft of updates to KDE applications and libraries, * A new Digikam release, which arrived in the ports tree on the day of its release. * A new KDevelop release arrived a day after its release. This update contains a number of crash fixes for refactoring support. * Qt was updated to Qt 5.15, the last in the Qt5 series and an LTS version. Bugfix releases are expected, but the next major Qt will be Qt 6. On the infrastructure front, August saw some minor updates to CMake and ninja. As usual, kde@ continues to support the work of xorg@ and gnome@ in maintaining the Free Desktop stack on FreeBSD, including XOrg, poppler, and xdg-utils. A new MAINTAINER group, desktop@, has been created to provide shared ownership of that shared stack. With Python2 deprecation looming, the build system for QtWebEngine -- itself a fork of Chromium -- is becoming a pressing issue in Q4 and will no doubt chew up a lot of time in the coming months. __________________________________________________________________ Documentation Noteworthy changes in the documentation tree, in manpages, or in external books/documents. DOCNG on FreeBSD Links DOCNG Website Repo=20 URL: https://gitlab.com/carlavilla/freebsd-hugo-website DOCNG Documentation Repo=20 URL: https://gitlab.com/carlavilla/freebsd-hugo-documentation DOCNG Share Repo=20 URL: https://gitlab.com/carlavilla/freebsd-hugo-data Contact: Sergio Carlavilla The Doc New Generation project aims to convert the website and all existing documentation to Hugo/AsciiDoctor. Right now almost everything is converted as you can see in the repositories. The objective of using Hugo and AsciiDoctor is to reduce the learning curve and let people to start quickly with our documentation system. Other benefits of using Hugo is that we can use other technologies aside from AsciiDoctor, like MarkDown, RST, Pandoc, etc. The remaining tasks include: * Finish the conversion of some books to AsciiDoctor. * Get some tweaks in the CSS to be responsive. * Add AsciiDoctor extensions to create an index of tables and figures. * Make a general review. The dates for making the migration have yet to be discussed. Patches, comments and objections are always welcome. __________________________________________________________________ Third-Party Projects Many projects build upon FreeBSD or incorporate components of FreeBSD into their project. As these projects may be of interest to the broader FreeBSD community, we sometimes include brief updates submitted by these projects in our quarterly report. The FreeBSD project makes no representation as to the accuracy or veracity of any claims in these submissions. Potluck - Flavour & Image Repository for pot Links Potluck Repository & Project=20 URL: https://potluck.honeyguide.net/ Potluck on github =20 URL: https://github.com/hny-gd/potluck pot project =20 URL: https://pot.pizzamig.dev Contact: <> pot is a jail management tool that also supports orchestration through nomad. Potluck aims to be to FreeBSD and pot what Dockerhub is to Linux and Docker: A repository of pot flavours and complete images for usage with pot. In the last quarter, an initial set of Nomad, Consul and Traefik images has been created that are sufficient to run a simple virtual datacenter out of the box. A three-part article series explaining how to set this up is also available now. Furthermore, ready-made images suitable for scheduling via Nomad and Consul in such an environment have been created, e.g. a BackupPC or a Postfix Backup MX service. Future plans include additional images and exposing more configuration options in the existing images to allow a more flexible usage. Beside general feedback and tests, additional flavours and patches are very welcome! Sponsors: Honeyguide GmbH & Honeyguide Group (Pty) Ltd ## Puppet Puppet Puppet's FreeBSD slack channel Bolt Choria Puppet Team puppet@FreeBSD.org Since out last status report a few years ago, the puppet@ team regularly updated the various Puppet ports to follow upstream releases of Puppet 4, Puppet 5 and Puppet 6. Puppet 4 was removed when it reached EOL. More recently, an effort was made to enhance Facter 4 so that it can be used as a drop-in replacement of Facter 3 on FreeBSD. Facter 4 is a Ruby rewrite of Facter 3, the C++ rewrite of Facter 2 which was initially in Ruby. As a consequence we have two ports for Facter: sysutils/facter is the C++ implementation (Facter 3) and sysutils/rubygems-facter is the Ruby implementation (updated from Facter 2 to Facter 4 a few weeks ago). The Puppet 5 and Puppet 6 ports already allow to choose which version of Facter to use. Facter 4 will be the default version of Facter with Puppet 7 which is expected to be released soon. We are getting ready to add a port for Puppet 7 as sysutils/puppet7 when it is available, along with PuppetServer 7 (sysutils/puppetserver7), and PuppetDB 7 (databases/puppetdb7). Regarding orchestration, most Marionette Collective ports have been deprecated for a long time, and the last component sysutils/mcollective is expected to be deprecated soon: Marionette Collective was not shipped anymore with Puppet 6 and Bolt has been made available as a lightweight replacement. Bolt is already available in the ports tree as sysutils/rubygems-bolt), but if you are using Marionette Collective, you are invited to look into Choria which will reach the ports tree soon as sysutils/choria. Choria is a direct evolution of Marionette Collective allowing a smooth transition from MCollective. Once Choria is available in the ports tree, Marionette Collective will be deprecated. __________________________________________________________________ News Home | Status Home Site Map | Legal Notices | =C2=A9 1995-2020 The FreeBSD Project. All rig= hts reserved. --u3hngvn2nte45ec7 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGTBAABCgB9FiEEDonNJPbg/JLIMoS6Ps5hSHzN87oFAl+QsRhfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDBF ODlDRDI0RjZFMEZDOTJDODMyODRCQTNFQ0U2MTQ4N0NDREYzQkEACgkQPs5hSHzN 87pGWQf6AvmDxeRYsxL7wxau9nX/Vlg7G1OcEVVhIuJaZAGb+zItXHsDjiqScFAF Q/DH+8wf2F719Sxdp2AA844cAm1FktjERua6ZmjZtGXx89OPpZf9YfEiexzBJshX rW/a7Vl2kNhB9768wT6PhjB/LrRWi2vq7C5oyKCncD+K10DbdZsGm4ImiBvM8Oxn xGTRNvkpisukm4Fta0idrGIvnJj3w9Nlen6mKCXWCcYugdEnBtpV4Y2JZM6qaW78 WCCkPekA+YLIhaWBe4bmRswMeztPntllBD90XBiDRpeJ6NTxgiDdGMN1s9b3WtOC D0YYpR7nJXRLGbZvHIMfBzMu2TbGOg== =7BFi -----END PGP SIGNATURE----- --u3hngvn2nte45ec7-- From owner-freebsd-stable@freebsd.org Thu Oct 22 13:39:33 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 27D35449215; Thu, 22 Oct 2020 13:39:33 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CH7kr2lBSz3dhg; Thu, 22 Oct 2020 13:39:32 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: by mail-ej1-x62b.google.com with SMTP id z5so2351883ejw.7; Thu, 22 Oct 2020 06:39:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Pe3+2DGM8XmVWFRV/bk0UIVlG2Luo4oi4g1YtTnKHRs=; b=s2C4Tl3utJXE1vsoR/psQTwqsc9+/Yko9bB4AGgQO8XkX6AJi/dw5Qcmn51Do7r2dy jXoyobaxqf8nElAxCgrmFGZg8UjL1Av5Lii5+hL0yNqV5xHMK5IyZDRgMlL1T0vadhgg y9F/rtfMaWkjJ/lwD+vw9aVuuIjUSWNuWLOOZX//jY8IdU8k1cJsA9+rDBx4AT7PXrha HQHjtSi56USB1viPNpM9IhnqJP4BfqwkHRwALgiuWkN6uo005U8L8VqFB3jnorD3CNS2 PyyKiVlY0qer1CZfXJS5UoSdClyJVhcfhgmWaKUkAwQDFsbI2pEtFHtwouhvVtJaq0DM qWMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Pe3+2DGM8XmVWFRV/bk0UIVlG2Luo4oi4g1YtTnKHRs=; b=nGUWgVUrPA4GcKB/fPO+BTpJ6KDBaH1SRhtgxDLHUNpj2NuAD1vFmb/CGVV09Oh8fN az1AR7LT/BLvthEgmS90f9XyMVec7vzDsYVSOfxwbIJrc2UIZxJhqI6hMqWEoYMzEtMs WMNbnuMh87nonmkmHzS0zaE6EFvzIjS7Ntb+HVheamaHqpZJj/l+OcEx3sCSrrS68Xse TYZBt/UGVcorOoAbj0u8yruaXLDwSuqY/1khN0G/HpVnMkro9fvRKPn/te7AffEr2H6d j7yg1YO9rZdlWbTXiGawuGCe53KwIwXaeiL91lV7FPTXrSWPfmKeLrByQ5RmmCKHWFe+ HTZg== X-Gm-Message-State: AOAM532qso3giQ0zAei3S/zJ0odBXplbZcmPWk1okFFEjYlJK1CRqx7y OlZ0C9dixwn3rVCj+BO/tpwe92p4t2h6hB5KFRtq0bUg X-Google-Smtp-Source: ABdhPJyTLmjpBzHJ5lDaDV4Qw0ZhakzCVCTgMhOwkY/WYALw8z6KS271l0BE4GzjxbU2WDQddOd4VluJ/wZsaCtnHxQ= X-Received: by 2002:a17:906:190b:: with SMTP id a11mr2317907eje.260.1603373970291; Thu, 22 Oct 2020 06:39:30 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Cassiano Peixoto Date: Thu, 22 Oct 2020 10:39:18 -0300 Message-ID: Subject: Re: Zpool doesn't boot anymore after FreeBSD 12.1 To: Andriy Gapon Cc: freebsd-stable@freebsd.org, freebsd-current@freebsd.org X-Rspamd-Queue-Id: 4CH7kr2lBSz3dhg X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=s2C4Tl3u; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of peixotocassiano@gmail.com designates 2a00:1450:4864:20::62b as permitted sender) smtp.mailfrom=peixotocassiano@gmail.com X-Spamd-Result: default: False [-2.72 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-0.99)[-0.992]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.004]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.28)[0.277]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62b:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable,freebsd-current]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2020 13:39:33 -0000 Hi Andriy, I've just tried copying my zfsloader from 11.2-STABLE (R350026) to FreeBSD 12.1 and 12.2 (STABLE) and fixed the issue. I also tried to use zfsloader of 11.3 but didn't work and the same issue happened. So it seems that something has changed on zfsloader after 11.2 that brings this issue. My question is: Should it be expected or is it a bug to be fixed? Thanks. On Thu, Oct 22, 2020 at 4:39 AM Andriy Gapon wrote: > On 21/10/2020 15:20, Cassiano Peixoto wrote: > > Hi there, > > > > Anyone can help please? I've many servers with this same issue. Thanks > > Can you try to replace /boot/zfsloader with zfsloader from other FreeBSD > versions? > E.g., 12.0, 12.2-RC, 11.4, recent snapshot of the CURRENT? > > > On Fri, Oct 16, 2020 at 10:24 AM Cassiano Peixoto < > peixotocassiano@gmail.com> > > wrote: > > > >> Hi there, > >> > >> I have a FreeBSD 12.1-STABLE running on VMWARE with one disk. Then I > added two more disks to expand my pool. BTW I already did it many time wi= th > no issues. > >> > >> I ran: > >> > >> # zpool status > >> pool: zroot > >> state: ONLINE > >> status: Some supported features are not enabled on the pool. The pool > can > >> still be used, but some features are unavailable. > >> action: Enable all features using 'zpool upgrade'. Once this is done, > >> the pool may no longer be accessible by software that does not > support > >> the features. See zpool-features(7) for details. > >> scan: none requested > >> config: > >> > >> NAME STATE READ WRITE CKSUM > >> zroot ONLINE 0 0 0 > >> gpt/disk0 ONLINE 0 0 0 > >> > >> errors: No known data errors > >> > >> # zpool add -f zroot da1 > >> # zpool add -f zroot da2 > >> # zpool status > >> pool: zroot > >> state: ONLINE > >> scan: none requested > >> config: > >> > >> NAME STATE READ WRITE CKSUM > >> zroot ONLINE 0 0 0 > >> gpt/disk0 ONLINE 0 0 0 > >> da1 ONLINE 0 0 0 > >> da2 ONLINE 0 0 0 > >> > >> errors: No known data errors > >> # reboot > >> > >> Then my system doesn=E2=80=99t boot anymore, i got the following error= : > >> > >> gptzfsboot: error 4 lba 2038346899 > >> gptzfsboot: error 4 lba 1361327267 > >> /boot/config: -Dh > >> > >> BTX loader 1.00 BTX version is 1.02 > >> Consoles: internal video/keyboard serial port > >> BIOS drive A: is fd0 > >> BIOS drive C: is disk0 > >> BIOS drive D: is disk1 > >> BIOS drive E: is disk2 > >> BIOS drive F: is disk3 > >> BIOS drive G: is disk4 > >> BIOS drive H: is disk5 > >> ZFS: i/o error - all block copies unavailable > >> ZFS: failed to read pool zroot directory object > >> BIOS 638kB/3143616kB available memory > >> > >> FreeBSD/x86 bootstrap loader, Revision 1.1 > >> ERROR: cannot open /boot/lua/loader.lua: invalid argument. > >> > >> Type '?' for list of commands, 'help' for more datailed help. > >> OK > >> > >> I can import my pool with no problems using the lived, but I could not > fix it. > >> > >> Seems a bug after 12.1-STABLE. Please, anyone can take a look ok that? > >> > >> Thanks. > >> > >> > >> > >> > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.or= g > " > > > > From owner-freebsd-stable@freebsd.org Thu Oct 22 16:11:24 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A221344C5BB for ; Thu, 22 Oct 2020 16:11:24 +0000 (UTC) (envelope-from michal.jakubik@zoho.com) Received: from sender4-op-o13.zoho.com (sender4-op-o13.zoho.com [136.143.188.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CHC634ctWz44lG for ; Thu, 22 Oct 2020 16:11:23 +0000 (UTC) (envelope-from michal.jakubik@zoho.com) ARC-Seal: i=1; a=rsa-sha256; t=1603383078; cv=none; d=zohomail.com; s=zohoarc; b=kyT9VK8HJOxE+iWyfLyXW32p27LYdN4vk7l+NmN0AS63ua6obuMcv9IgZvnoNPgYhICKXwXzzmnntG3+SItfI7C/5ioQZn/nBJiSkXEh2YFrL6TB9Hp+jT2vyoO64dJ5EpjwaN4ebBA02doWIgVO3EBQpvv6dhuvrSpBLjW8dXU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1603383078; h=Content-Type:Date:From:MIME-Version:Message-ID:Reply-To:Subject:To; bh=DJ1WCs9fcoRIieS1P21/uNbTWIoY5RyTTsxAfuaIu1Y=; b=QCfQaUjw+cOMfFmMtAjzXHELRZUv378gZ26fcOiTGXWWyrwivW0X/96V/nYbBoQbzx/TGV8djU9Qy3zDU034ea1MnlQt7X7zOinNqMNgljAlhy6uJzxRdrcGOkuEGQOi1frYgpA8SJDMr7rMKKgD+OTLZSZZS+ZqHfSlqFkx5XY= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass smtp.mailfrom=michal.jakubik@zoho.com; dmarc=pass header.from= header.from= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=zapps768; d=zoho.com; h=date:from:reply-to:to:message-id:in-reply-to:subject:mime-version:content-type:user-agent; b=TCpagYD8T8Pt/NToReUxJD1yAlTeKu9OgdiA5F4sh6TvFvNod85ncafU0KaQtzVMxghSpKPXmrQ+ V4N+ICoKX6vBayqlUJf25IZgJO5Mt48blVlZkWjB+qJqeNxMZM2E Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1603383077372177.1909081781797; Thu, 22 Oct 2020 09:11:17 -0700 (PDT) Date: Thu, 22 Oct 2020 12:11:17 -0400 From: Mike Jakubik Reply-To: mike.jakubik@swiftsmsgateway.com To: "Freebsd Stable" Message-Id: <175511429f5.ff83169625753.7800820802617382912@swiftsmsgateway.com> In-Reply-To: Subject: Custom kernel breaks py-glances in 12.2 MIME-Version: 1.0 Importance: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail X-Rspamd-Queue-Id: 4CHC634ctWz44lG X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of michal.jakubik@zoho.com designates 136.143.188.13 as permitted sender) smtp.mailfrom=michal.jakubik@zoho.com X-Spamd-Result: default: False [-2.89 / 15.00]; HAS_REPLYTO(0.00)[mike.jakubik@swiftsmsgateway.com]; XM_UA_NO_VERSION(0.01)[]; RWL_MAILSPIKE_GOOD(0.00)[136.143.188.13:from]; R_SPF_ALLOW(-0.20)[+ip4:136.143.188.0/24]; REPLYTO_ADDR_EQ_FROM(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.02)[-0.017]; FORGED_SENDER(0.30)[mike.jakubik@swiftsmsgateway.com,michal.jakubik@zoho.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[zoho.com]; MID_RHS_MATCH_FROM(0.00)[]; FROM_NEQ_ENVFROM(0.00)[mike.jakubik@swiftsmsgateway.com,michal.jakubik@zoho.com]; ASN(0.00)[asn:2639, ipnet:136.143.188.0/23, country:US]; NEURAL_HAM_MEDIUM(-0.90)[-0.905]; ARC_ALLOW(-1.00)[zohomail.com:s=zohoarc:i=1]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.98)[-0.978]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[swiftsmsgateway.com]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[136.143.188.13:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2020 16:11:24 -0000 Hi, When i compile a non GENERIC kernel (which mainly removes some debugging an= d devices i do not have) the Python program glances fails with the followin= g message. [root@jailer ~]# glances=20 Traceback (most recent call last): =C2=A0 File "/usr/local/bin/glances", line 8, in =C2=A0=C2=A0=C2=A0 sys.exit(main()) =C2=A0 File "/usr/local/lib/python3.9/site-packages/glances/__init__.py", l= ine 143, in main =C2=A0=C2=A0=C2=A0 start(config=3Dconfig, args=3Dargs) =C2=A0 File "/usr/local/lib/python3.9/site-packages/glances/__init__.py", l= ine 108, in start =C2=A0=C2=A0=C2=A0 mode =3D GlancesMode(config=3Dconfig, args=3Dargs) =C2=A0 File "/usr/local/lib/python3.9/site-packages/glances/standalone.py",= line 78, in __init__ =C2=A0=C2=A0=C2=A0 self.stats.update() =C2=A0 File "/usr/local/lib/python3.9/site-packages/glances/stats.py", line= 234, in update =C2=A0=C2=A0=C2=A0 self._plugins[p].update() =C2=A0 File "/usr/local/lib/python3.9/site-packages/glances/plugins/glances= _plugin.py", line 925, in wrapper =C2=A0=C2=A0=C2=A0 ret =3D fct(self, *args, **kw) =C2=A0 File "/usr/local/lib/python3.9/site-packages/glances/plugins/glances= _plugin.py", line 935, in wrapper =C2=A0=C2=A0=C2=A0 ret =3D fct(*args, **kw) =C2=A0 File "/usr/local/lib/python3.9/site-packages/glances/plugins/glances= _cpu.py", line 84, in update =C2=A0=C2=A0=C2=A0 stats =3D self.update_local() =C2=A0 File "/usr/local/lib/python3.9/site-packages/glances/plugins/glances= _cpu.py", line 118, in update_local =C2=A0=C2=A0=C2=A0 cpu_stats =3D psutil.cpu_stats() =C2=A0 File "/usr/local/lib/python3.9/site-packages/psutil/__init__.py", li= ne 1866, in cpu_stats =C2=A0=C2=A0=C2=A0 return _psplatform.cpu_stats() =C2=A0 File "/usr/local/lib/python3.9/site-packages/psutil/_psbsd.py", line= 293, in cpu_stats =C2=A0=C2=A0=C2=A0 ctxsw, intrs, soft_intrs, syscalls, traps =3D cext.cpu_s= tats() OSError: [Errno 12] Cannot allocate memory (originated from sysctlbyname('v= m.stats.sys.v_soft')) [root@jailer ~]# sysctl vm.stats.sys.v_soft vm.stats.sys.v_soft: 3101 [root@jailer ~]# uname -a=20 FreeBSD jailer.local 12.2-STABLE FreeBSD 12.2-STABLE r366936 JAILER=C2=A0 a= md64 I don't understand what is breaking it as everything else works fine and i = only removed the usual stuff i do. Below is the custom kernel config. Thanks. # $FreeBSD: stable/12/sys/amd64/conf/GENERIC 365612 2020-09-10 20:46:16Z er= j $=20 cpu=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= HAMMER ident=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 JAILER options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 SCHED_ULE=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= # ULE scheduler options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 NUMA=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 # Non-Uniform Memory Architecture support options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 PREEMPTION=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Ena= ble kernel thread preemption options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 VIMAGE=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 # Subsystem virtualization, e.g. VNET options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 INET=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 # InterNETworking options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 INET6=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 # IPv6 communications protocols options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 IPSEC=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 # IP (v4/v6) security options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 IPSEC_SUPPORT=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Allow kldload of i= psec and tcpmd5 options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 TCP_OFFLOAD=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # TCP of= fload options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 TCP_BLACKBOX=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Enhanced TCP= event logging options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 TCP_HHOOK=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= # hhook(9) framework for TCP options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 TCP_RFC7413=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # TCP Fa= st Open options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 SCTP=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 # Stream Control Transmission Protocol options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 FFS=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Berkeley Fast Filesystem options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 SOFTUPDATES=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Enable= FFS soft updates support options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 UFS_ACL=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 # Support for access control lists options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 UFS_DIRHASH=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Improv= e performance on big directories options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 PROCFS=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 # Process filesystem (requires PSEUDOFS) options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 PSEUDOFS=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 # Pseudo-filesystem framework options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 GEOM_LABEL=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Pro= vides labelization options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 EFIRT=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 # EFI Runtime Services support options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 SCSI_DELAY=3D5000= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Delay (in ms) before pro= bing SCSI options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 KTRACE=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 # ktrace(1) support options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 STACK=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 # stack(9) support options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 SYSVSHM=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 # SYSV-style shared memory options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 SYSVMSG=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 # SYSV-style message queues options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 SYSVSEM=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 # SYSV-style semaphores options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 _KPOSIX_PRIORITY_SC= HEDULING # POSIX P1003_1B real-time extensions options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 PRINTF_BUFR_SIZE=3D= 128=C2=A0=C2=A0=C2=A0 # Prevent printf output being interspersed. options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 KBD_INSTALL_CDEV=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # install a CDEV entry in /dev options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 HWPMC_HOOKS=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Necess= ary kernel hooks for hwpmc(4) options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 AUDIT=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 # Security event auditing options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 CAPABILITY_MODE=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Capsicum capability mode options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 CAPABILITIES=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Capsicum cap= abilities options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 MAC=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # TrustedBSD MAC Framework options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 DDB_CTF=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 # Kernel ELF linker loads CTF data options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 INCLUDE_CONFIG_FILE= =C2=A0=C2=A0=C2=A0=C2=A0 # Include this file in kernel options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 RACCT=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 # Resource accounting framework options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 RACCT_DEFAULT_TO_DI= SABLED # Set kern.racct.enable=3D0 by default options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 RCTL=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 # Resource limits # Make an SMP-capable kernel by default options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 SMP=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Symmetric MultiProcessor Kernel options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 EARLY_AP_STARTUP # Bus support. device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 acpi options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ACPI_DMAR device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pci options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 PCI_HP=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 # PCI-Express native HotPlug options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 PCI_IOV=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 # PCI SR-IOV support # ATA controllers device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ahci=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # AHCI-compatible SATA controllers device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ata=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Legacy ATA/SATA controllers # ATA/SCSI peripherals device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 scbus=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 # SCSI bus (required for ATA/SCSI) device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 da=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Direct Access (disks) device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 cd=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # CD device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Passthrough device (direct ATA/SCSI access= ) device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ses=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Enclosure Services (SES and SAF-TE) #device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ctl=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # CAM Target Layer # RAID controllers interfaced to the SCSI subsystem device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 amr=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # AMI MegaRAID device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ciss=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Compaq Smart RAID 5* # atkbdc0 controls both the keyboard and the PS/2 mouse device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 atkbdc=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 # AT keyboard controller device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 atkbd=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 # AT keyboard device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 psm=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # PS/2 mouse device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 kbdmux=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 # keyboard multiplexer device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 vga=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # VGA video card driver options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 VESA=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 # Add support for VESA BIOS Extensions (VBE) device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 splash=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 # Splash screen and screen saver support # syscons is the default console driver, resembling an SCO console device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 sc options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 SC_PIXEL_MODE=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # add support for th= e raster text mode # vt is the new video console driver device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 vt device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 vt_vga device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 vt_efifb # Serial (COM) ports device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 uart=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Generic UART driver # PCI Ethernet NICs that use the common MII bus controller code. # NOTE: Be sure to keep the 'device miibus' line in order to use these NICs= ! device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 miibus=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 # MII bus support device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 bce=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Broadcom BCM5706/BCM5708 Gigabit Eth= ernet # Pseudo devices. device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 crypto=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 # core crypto support device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 loop=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Network loopback device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 random=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 # Entropy device device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 padlock_rng=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # VIA= Padlock RNG device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 rdrand_rng=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= # Intel Bull Mountain RNG device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ether=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 # Ethernet support device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 vlan=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # 802.1Q VLAN support device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 tuntap=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 # Packet tunnel. device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 md=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Memory "disks" device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 gif=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # IPv6 and IPv4 tunneling device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 firmware=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 # firmware assist module device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 cryptodev device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 aesni # The `bpf' device enables the Berkeley Packet Filter. # Be aware of the administrative consequences of enabling this! # Note that 'bpf' is required for DHCP. device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 bpf=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Berkeley packet filter # USB support options=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 USB_DEBUG=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= # enable debug msgs device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 uhci=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # UHCI PCI->USB interface device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ohci=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # OHCI PCI->USB interface device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ehci=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # EHCI PCI->USB interface (USB 2.0) device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 xhci=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # XHCI PCI->USB interface (USB 3.0) device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 usb=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # USB Bus (required) device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ukbd=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Keyboard device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 umass=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 # Disks/Mass storage - Requires scbus and da # Netmap provides direct access to TX/RX rings on supported NICs device=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 netmap=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 # netmap(4) support From owner-freebsd-stable@freebsd.org Thu Oct 22 17:37:51 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6535C44DB89; Thu, 22 Oct 2020 17:37:51 +0000 (UTC) (envelope-from sergey.dyatko@gmail.com) Received: from mail-lj1-x244.google.com (mail-lj1-x244.google.com [IPv6:2a00:1450:4864:20::244]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CHF1p4694z4Cfm; Thu, 22 Oct 2020 17:37:50 +0000 (UTC) (envelope-from sergey.dyatko@gmail.com) Received: by mail-lj1-x244.google.com with SMTP id h20so2818611lji.9; Thu, 22 Oct 2020 10:37:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MhoOwATKEWNb+CUubFE0qpbga7DCwAufLRF1GI67eHc=; b=smFfopyybvrzr+pmaUkf7IBdThKftSCsAlzXiQT4VI49loRSNj8E5GNtlMTOjrQtiJ j+aQmm7ds1MsAtjZ9C1WG1Q7tMEa3O/TC+BoMDq7gsv4+F6tBVQZev3H+Y5+VvIBRP46 ecVlCuf65QlfmX9KdunidkRPBb5RJ3P6HQV4zhcKU9W3oWyI8tftE3Dt5yi6GNYWm2gZ Embty9ZbYaAJwwbc2mu/Qi4XAmdrS/3a/BQ0QaztmW/jDRigemx44n12T5u9+7I+Tw+p MGxN3rQDHuF9/YZaXgTdeTIjS+FUOnbb8DB4zkhlD8tpltGBTTTMG88Ci53WWstUQ50w BuBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MhoOwATKEWNb+CUubFE0qpbga7DCwAufLRF1GI67eHc=; b=DVQylvZoymL3QvqhB5znWkJfQKqgi5MU6pI7/P+Fw3s8OVxFu6D/R9GdLpF0rVxLLy TJJUT4fUtbHyCVeWi9qq4ksHb/6k6BBnHjG7qzbjRHNJM110f92dTTpW37CCeRYwXLic sbcHZaVfKzTbTHonymcogvW9fsSkOKAmQUkBJIOgn5M3F9qSV6pPY+blVI1PMUpA0LkP G8+a9kSi1t0GZ7Evn0h4ap/WonqyLXUehAU5ztYcEZ5S1joCgWcSphLaQdedSjDKAQzS Y6Rukut1E4zFiwqqwxET3waWjz7IEirNH5r1IXH0m5Jbd8saNpIXryyXGR1Nwoa9upqW t2wQ== X-Gm-Message-State: AOAM5339QmtO1nF56RFiDZkJS3AGfoO7EcO/ZHbJ0rwFSAVeGsRCYgzr CS4sfQqVviOvqv4hHIcslrqXm47vs40= X-Google-Smtp-Source: ABdhPJzPieafJ/G5sump7HcpCI+vTgHuAIgI5rigXQEJtUGEy+kBKuOu1D/yawjJoq7RenHeU2PouQ== X-Received: by 2002:a2e:6e14:: with SMTP id j20mr1301609ljc.361.1603388268743; Thu, 22 Oct 2020 10:37:48 -0700 (PDT) Received: from laptop.domain (mm-223-241-125-178.mfilial.dynamic.pppoe.byfly.by. [178.125.241.223]) by smtp.gmail.com with ESMTPSA id j12sm388880ljg.22.2020.10.22.10.37.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Oct 2020 10:37:48 -0700 (PDT) Date: Thu, 22 Oct 2020 20:37:52 +0300 From: "Sergey V. Dyatko" To: freebsd-current@freebsd.org Cc: freebsd-stable@freebsd.org Subject: Re: Zpool doesn't boot anymore after FreeBSD 12.1 Message-ID: <20201022203752.36fb6f3f@laptop.domain> In-Reply-To: <33381216-498a-51ff-839c-da772c43b60e@FreeBSD.org> References: <33381216-498a-51ff-839c-da772c43b60e@FreeBSD.org> X-Mailer: Claws Mail 3.17.5 (GTK+ 2.24.32; amd64-portbld-freebsd13.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4CHF1p4694z4Cfm X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=smFfopyy; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of sergeydyatko@gmail.com designates 2a00:1450:4864:20::244 as permitted sender) smtp.mailfrom=sergeydyatko@gmail.com X-Spamd-Result: default: False [-3.36 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.41)[-0.406]; RECEIVED_SPAMHAUS_PBL(0.00)[178.125.241.223:received]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.97)[-0.966]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.99)[-0.987]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::244:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable,freebsd-current] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2020 17:37:51 -0000 On Thu, 22 Oct 2020 16:42:16 +0300 Andriy Gapon wrote: > On 22/10/2020 16:39, Cassiano Peixoto wrote: > > Hi Andriy, > > > > I've just tried copying my zfsloader from 11.2-STABLE (R350026) to FreeBSD > > 12.1 and 12.2 (STABLE) and fixed the issue. > > > > I also tried to use zfsloader of 11.3 but didn't work and the same issue > > happened. > > > > So it seems that something has changed on zfsloader after 11.2 that brings > > this issue. > > > > My question is: Should it be expected or is it a bug to be fixed? > > > > In my opinion it's a bug. > zfsloader should not require that disks must be partitioned. > +1 That's why I have terribly outdated 13-CURRENT on bunch of servers and can't update them. Most of them look like this: [tiger@st25]:~>zpool status pool: st25 state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM st25 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 ada0 ONLINE 0 0 0 ada1 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 ada2 ONLINE 0 0 0 ada3 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 ada4 ONLINE 0 0 0 ada5 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 ada6 ONLINE 0 0 0 ada7 ONLINE 0 0 0 errors: No known data errors They haven't separate boot device and have from 7 to 30+ Tb of data that I can't backup anywhere for re-install server with disk partitioning. Some time ago I managed to get aroud this reverting 2 commits ( r342151 + don't remember revision) but then there where some other improvements after which I lost the opportunity to revert this commits -- wbr, Sergey From owner-freebsd-stable@freebsd.org Thu Oct 22 18:05:08 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 689F044EFF8; Thu, 22 Oct 2020 18:05:08 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: from mail-ej1-x641.google.com (mail-ej1-x641.google.com [IPv6:2a00:1450:4864:20::641]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CHFdH3HbHz4Kjr; Thu, 22 Oct 2020 18:05:07 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: by mail-ej1-x641.google.com with SMTP id h24so3610922ejg.9; Thu, 22 Oct 2020 11:05:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ilEW49fER+qVjBWTrmfdKypOKYTvj7QU1vKeI1FiKYA=; b=u2KMrxKJb05GTJYLeOryFIBJuwjP2wizBTjFXuIsmv4cTpW/OhSPl+QSi/T0NhX/Cb e3lCZPpI0AzrNJEtF04X0oZh8rscSGLKZ782l4tErbfTVXKfALWIoNJKejCHTD8RCHyO MtPgFlZI8wvuUT1NtFAGIyNhlRAJrT8NTd0zVfLc1PbLAZk3uirLslcmXGA+WcWBp3TY 8wgS8mmdOz5ZCRoanssjcwgU/LQpY8e703R0jbqk6KVUrXXbFMA6cVkUYlnFBMUm6u1f ILkI6KgaL7X8Y2BnxXDT5NdDS5E3FTyg6FYDgAv1kT8kHZKlF/6UHoGdwmOVzaAEhQ34 ahlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ilEW49fER+qVjBWTrmfdKypOKYTvj7QU1vKeI1FiKYA=; b=nQdOQXBScvPacjDqIL9k4aqkwINJVUAIzJdIRH9YsJFmpzTVC315+2ZnEb1vJecDL1 fPTyziZdaYj6ggFBgs/qfNs8AxVBgZxLGoSBhImV5Y1v4zyEiLtcDIvZKGPNoDqUk1wr /HWa0funsDae51L1ZRLXx7jLlFmG8GPyhv33AKEwipA5YAKBM5vn6B0oCGd1JHMYF7OZ 2y9kkE/bUFI4sWgy/UO1E4GU9L53ZAP1MQmvCRljqM4Kf4PeD5YHD3N5/ekdBih4uH77 MS86fJmNDSHsvL2TI3rDAqhMhwtme6W5Dakzf/hxs3ejKZAhonGecI/vhmHwpTYPRoiJ JVAA== X-Gm-Message-State: AOAM5303vIQcztoKJxJnOlN2OXcEmSwi3iifCQ9t3qjXGFu/rLjQL4R7 rkfUjgsyU3rLWjhvt0rBQLJhRUP49177J8qbPYE= X-Google-Smtp-Source: ABdhPJwcjdCZYD88oQ7GI56i3MqnwUtXT2a+QYFxjCFeo4/1WHmwEIHqoWbnSR+EaD8dQ+XQMAScmybLwML8KcvhBOI= X-Received: by 2002:a17:906:3799:: with SMTP id n25mr3406390ejc.6.1603389906132; Thu, 22 Oct 2020 11:05:06 -0700 (PDT) MIME-Version: 1.0 References: <33381216-498a-51ff-839c-da772c43b60e@FreeBSD.org> <20201022203752.36fb6f3f@laptop.domain> In-Reply-To: <20201022203752.36fb6f3f@laptop.domain> From: Cassiano Peixoto Date: Thu, 22 Oct 2020 15:04:54 -0300 Message-ID: Subject: Re: Zpool doesn't boot anymore after FreeBSD 12.1 To: "Sergey V. Dyatko" Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org X-Rspamd-Queue-Id: 4CHFdH3HbHz4Kjr X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=u2KMrxKJ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of peixotocassiano@gmail.com designates 2a00:1450:4864:20::641 as permitted sender) smtp.mailfrom=peixotocassiano@gmail.com X-Spamd-Result: default: False [-3.12 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-0.96)[-0.959]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::641:from]; NEURAL_HAM_SHORT(-0.16)[-0.158]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-stable]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2020 18:05:08 -0000 Sergey, I agree with you. The best thing is to revert the commit on zfsloader. Many people didn't realize this issue yet, but they will run into a big problem soon. On Thu, Oct 22, 2020 at 2:41 PM Sergey V. Dyatko wrote: > On Thu, 22 Oct 2020 16:42:16 +0300 > Andriy Gapon wrote: > > > On 22/10/2020 16:39, Cassiano Peixoto wrote: > > > Hi Andriy, > > > > > > I've just tried copying my zfsloader from 11.2-STABLE (R350026) to > FreeBSD > > > 12.1 and 12.2 (STABLE) and fixed the issue. > > > > > > I also tried to use zfsloader of 11.3 but didn't work and the same > issue > > > happened. > > > > > > So it seems that something has changed on zfsloader after 11.2 that > brings > > > this issue. > > > > > > My question is: Should it be expected or is it a bug to be fixed? > > > > > > > In my opinion it's a bug. > > zfsloader should not require that disks must be partitioned. > > > > +1 > That's why I have terribly outdated 13-CURRENT on bunch of servers and > can't > update them. > > Most of them look like this: > [tiger@st25]:~>zpool status > pool: st25 > state: ONLINE > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > st25 ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > ada0 ONLINE 0 0 0 > ada1 ONLINE 0 0 0 > mirror-1 ONLINE 0 0 0 > ada2 ONLINE 0 0 0 > ada3 ONLINE 0 0 0 > mirror-2 ONLINE 0 0 0 > ada4 ONLINE 0 0 0 > ada5 ONLINE 0 0 0 > mirror-3 ONLINE 0 0 0 > ada6 ONLINE 0 0 0 > ada7 ONLINE 0 0 0 > > errors: No known data errors > > They haven't separate boot device and have from 7 to 30+ Tb of data that I > can't backup anywhere for re-install server with disk partitioning. Some > time > ago I managed to get aroud this reverting 2 commits ( r342151 + don't > remember revision) but then there where some other improvements after > which I > lost the opportunity to revert this commits > > -- > wbr, Sergey > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Thu Oct 22 20:29:49 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E587D42A0BB; Thu, 22 Oct 2020 20:29:49 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: from mail-ed1-x543.google.com (mail-ed1-x543.google.com [IPv6:2a00:1450:4864:20::543]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CHJrD6xvPz4T30; Thu, 22 Oct 2020 20:29:48 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: by mail-ed1-x543.google.com with SMTP id w23so3091668edl.0; Thu, 22 Oct 2020 13:29:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=hSdEecR2NGI1Upy26BcJd9QBMN1LddL+WdOnAWR6fFk=; b=rWwIU5Qv0sWHASqIv1rFopzCV/pnT3/wLklcL2CpnmoPwzuAdBCSgNJDWZUf84JhS8 km4zOrAQhZpfKRqsdu8sfNmCwit0aF6v7QTN5rEwOleeDpNQcv2r55abibZIZ5lyQpwy Xgr7UK3dEU89ymGVm4iOuOYGJJIyqssFNA8RfZBuy0Vi+z9ll3+zg1+v2Mk6Anrs/uin x6lXD00TBLlF2Vph5bLVUUVczncnANy5xngya5mNbXJG4/n5rlS2TCS8Mk/Rpvlfpuja B08mPh45cZp6SuYNZyUohS7ppEE6DRs/ilzuz18Py27dZ9EqmIpmZwhUHeLxNJ2tzvnw Hfvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=hSdEecR2NGI1Upy26BcJd9QBMN1LddL+WdOnAWR6fFk=; b=eFJ5F3i/zXfzWmULTjFNJcMTWfR9xa9VOQRmJT4grF6Shu2VWBDVTeDeTjL01QdS/c zDUIwmfhF/jzI4GMyujqnCUmeo5qs6DXTWgMQUSEQy2ZToyCH9r0U8cJpIWuewaxq7tn LGY/bWJIrMbehMqkr1hq4ZFoh3WTwyez0YOqSn+RWsHPqjtw2KujrE4lJCqNspBeK71L R345h7V/B+0DWcqHAQplFppGuRz1IPa2OcFgdAOx28P8TUYJHO0pbajkisICEJpQ3hd5 xwAvkI5sbsTkqJAde4q8nEDvCaC30WMz1VuJJNKlMtl3QQGWdZLXaFg+SUWNMPx76SwS 1Iiw== X-Gm-Message-State: AOAM531z5ijvHJrPhQIvQz3SvwzeF7Woaf9mp7DwyjbjoBGVL/gitozL 9oN5NxwTHVKFVmohdraPdA5XZujjbnJj/CpQuvI= X-Google-Smtp-Source: ABdhPJza6bQullddDHOTl4OLMuYUvhDxWKlcLpdxgMvWCvMoNIrL3TgQecGoxqzF9bwzNm7dut4ZlxMYB9/GLzbm9wk= X-Received: by 2002:a05:6402:c12:: with SMTP id co18mr3824768edb.162.1603398587360; Thu, 22 Oct 2020 13:29:47 -0700 (PDT) MIME-Version: 1.0 References: <33381216-498a-51ff-839c-da772c43b60e@FreeBSD.org> <20201022203752.36fb6f3f@laptop.domain> In-Reply-To: From: Cassiano Peixoto Date: Thu, 22 Oct 2020 17:29:35 -0300 Message-ID: Subject: Re: Zpool doesn't boot anymore after FreeBSD 12.1 To: "Sergey V. Dyatko" , tsoome@me.com Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org X-Rspamd-Queue-Id: 4CHJrD6xvPz4T30 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=rWwIU5Qv; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of peixotocassiano@gmail.com designates 2a00:1450:4864:20::543 as permitted sender) smtp.mailfrom=peixotocassiano@gmail.com X-Spamd-Result: default: False [-2.02 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.57)[-0.569]; FREEMAIL_TO(0.00)[gmail.com,me.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.95)[-0.945]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.01)[-1.007]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::543:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; SUSPICIOUS_RECIPS(1.50)[]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-stable] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2020 20:29:50 -0000 Hi Toomas, Thank you for this patch. Can I know when it will be MFCed? I'd like to try. Hi! Please try 366951:) I think, it should get things better for you. rgds, toomas On Thu, Oct 22, 2020 at 3:04 PM Cassiano Peixoto wrote: > Sergey, > > I agree with you. The best thing is to revert the commit on zfsloader. > Many people didn't realize this issue yet, but they will run into a big > problem soon. > > On Thu, Oct 22, 2020 at 2:41 PM Sergey V. Dyatko > wrote: > >> On Thu, 22 Oct 2020 16:42:16 +0300 >> Andriy Gapon wrote: >> >> > On 22/10/2020 16:39, Cassiano Peixoto wrote: >> > > Hi Andriy, >> > > >> > > I've just tried copying my zfsloader from 11.2-STABLE (R350026) to >> FreeBSD >> > > 12.1 and 12.2 (STABLE) and fixed the issue. >> > > >> > > I also tried to use zfsloader of 11.3 but didn't work and the same >> issue >> > > happened. >> > > >> > > So it seems that something has changed on zfsloader after 11.2 that >> brings >> > > this issue. >> > > >> > > My question is: Should it be expected or is it a bug to be fixed? >> > > >> > >> > In my opinion it's a bug. >> > zfsloader should not require that disks must be partitioned. >> > >> >> +1 >> That's why I have terribly outdated 13-CURRENT on bunch of servers and >> can't >> update them. >> >> Most of them look like this: >> [tiger@st25]:~>zpool status >> pool: st25 >> state: ONLINE >> scan: none requested >> config: >> >> NAME STATE READ WRITE CKSUM >> st25 ONLINE 0 0 0 >> mirror-0 ONLINE 0 0 0 >> ada0 ONLINE 0 0 0 >> ada1 ONLINE 0 0 0 >> mirror-1 ONLINE 0 0 0 >> ada2 ONLINE 0 0 0 >> ada3 ONLINE 0 0 0 >> mirror-2 ONLINE 0 0 0 >> ada4 ONLINE 0 0 0 >> ada5 ONLINE 0 0 0 >> mirror-3 ONLINE 0 0 0 >> ada6 ONLINE 0 0 0 >> ada7 ONLINE 0 0 0 >> >> errors: No known data errors >> >> They haven't separate boot device and have from 7 to 30+ Tb of data that I >> can't backup anywhere for re-install server with disk partitioning. Some >> time >> ago I managed to get aroud this reverting 2 commits ( r342151 + don't >> remember revision) but then there where some other improvements after >> which I >> lost the opportunity to revert this commits >> >> -- >> wbr, Sergey >> >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >> > From owner-freebsd-stable@freebsd.org Thu Oct 22 20:34:50 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DD1EE42A5D9; Thu, 22 Oct 2020 20:34:50 +0000 (UTC) (envelope-from sergey.dyatko@gmail.com) Received: from mail-ed1-x542.google.com (mail-ed1-x542.google.com [IPv6:2a00:1450:4864:20::542]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CHJy21Kwfz4TWZ; Thu, 22 Oct 2020 20:34:49 +0000 (UTC) (envelope-from sergey.dyatko@gmail.com) Received: by mail-ed1-x542.google.com with SMTP id x1so3099559eds.1; Thu, 22 Oct 2020 13:34:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iRVBUuQ+c16gnVnu5vNwj1ts2E/pLp1XhrYJ1I3nDpY=; b=WvnxgnBIj8FvxaIlNO2b5zolmGWHX4+AFs/r7FMcSRCTMvoTzlU9ugMQxvMnOBxaEz AxigZIB5mOY6J9BvE1S7gCN2btjvYCdKhx/SmzGFbvNCaZrNufvpExyLXLEeGMq/UGBn FKQxXOkDdOg43uriHXtPsRRIY+44IKMLC5CCmYNLtmZlYEbtSgvThiK5NWNf9YZSEyJx G0NIa3JjMrKD5alhilib8XPbLRyRnHnYWlJUtbMuyV+KQWDFlHyzY2eiGI4ljVXQiMoL uDVNbrJs7LNM3I6kNwcSvPs/rzHjHgQRU5b87daZDP0gb2zCKHLGHUHT72u7FPF6XZvi vBZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iRVBUuQ+c16gnVnu5vNwj1ts2E/pLp1XhrYJ1I3nDpY=; b=a2vyPBsZZe1OmNR0CKSgFgjR5HPdPHTdbDPbgqhb+K1mEREi1iNjtzb6fC8Ky9gFhQ 1pVMzNqLjC3cYPW/NDzmlhMWCRttUP+OVmuSGSvso/4VKuNs1Y1lQQkqajIpAJTC9MtX dyC3ca6BnCimzFFrsEfHFgEbQSOCD/ZSSv1N8bDlfNN3f9U6j+xBpHmmVK+9YZETwdv3 cdLQaaeChgn6ptkUCDy14WGkibfWM3tbOIDUZNToO76rCriSpnlikYcKhC7iDIDkqiPM CRFPf3tQcyjCQNFPCijQr18bzO+bXFdpcOtDC1yIrit9Ar/Ck/eAzrzaQPQYn094Sr3j OpJQ== X-Gm-Message-State: AOAM531Yw/Q+Wv3tFCJGdx+zN//qZuSf8Wymqx1aX7R8gjDbg7fzLOg9 QEoZGrslkvqPgSC166mR0jbPwTYkHac= X-Google-Smtp-Source: ABdhPJywIuMU1JC3EzVnOdMU6W9lHKz9AIXLQVbdIs5dr/cgu/8e582YE2SY6MSDlYPoX3mKzEVfdA== X-Received: by 2002:aa7:cc10:: with SMTP id q16mr3139205edt.96.1603398888672; Thu, 22 Oct 2020 13:34:48 -0700 (PDT) Received: from laptop.domain (mm-223-241-125-178.mfilial.dynamic.pppoe.byfly.by. [178.125.241.223]) by smtp.gmail.com with ESMTPSA id p16sm1385726ejz.103.2020.10.22.13.34.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Oct 2020 13:34:48 -0700 (PDT) Date: Thu, 22 Oct 2020 23:34:51 +0300 From: "Sergey V. Dyatko" To: Toomas Soome Cc: FreeBSD Current , freebsd-stable@freebsd.org Subject: Re: Zpool doesn't boot anymore after FreeBSD 12.1 Message-ID: <20201022233451.40a3aed6@laptop.domain> In-Reply-To: <2E4B5AFC-C23E-4F33-8EFA-D285A998A4F0@me.com> References: <33381216-498a-51ff-839c-da772c43b60e@FreeBSD.org> <20201022203752.36fb6f3f@laptop.domain> <2E4B5AFC-C23E-4F33-8EFA-D285A998A4F0@me.com> X-Mailer: Claws Mail 3.17.5 (GTK+ 2.24.32; amd64-portbld-freebsd13.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4CHJy21Kwfz4TWZ X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=WvnxgnBI; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of sergeydyatko@gmail.com designates 2a00:1450:4864:20::542 as permitted sender) smtp.mailfrom=sergeydyatko@gmail.com X-Spamd-Result: default: False [-3.32 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.34)[-0.341]; FREEMAIL_TO(0.00)[me.com]; RECEIVED_SPAMHAUS_PBL(0.00)[178.125.241.223:received]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.97)[-0.973]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.004]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::542:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2020 20:34:50 -0000 On Thu, 22 Oct 2020 23:04:09 +0300 Toomas Soome wrote: > Hi! > > Please try 366951:) I think, it should get things better for you. > > rgds, > toomas > Thanks! I'll try as soon as I can find 7-8Tb for backup :) btw, can I update my head@r324614 to 366951 at once ? > > On 22. Oct 2020, at 20:37, Sergey V. Dyatko wrote: > > > > On Thu, 22 Oct 2020 16:42:16 +0300 > > Andriy Gapon wrote: > > > >> On 22/10/2020 16:39, Cassiano Peixoto wrote: > >>> Hi Andriy, > >>> > >>> I've just tried copying my zfsloader from 11.2-STABLE (R350026) to FreeBSD > >>> 12.1 and 12.2 (STABLE) and fixed the issue. > >>> > >>> I also tried to use zfsloader of 11.3 but didn't work and the same issue > >>> happened. > >>> > >>> So it seems that something has changed on zfsloader after 11.2 that brings > >>> this issue. > >>> > >>> My question is: Should it be expected or is it a bug to be fixed? > >>> > >> > >> In my opinion it's a bug. > >> zfsloader should not require that disks must be partitioned. > >> > > > > +1 > > That's why I have terribly outdated 13-CURRENT on bunch of servers and can't > > update them. > > > > Most of them look like this: > > [tiger@st25]:~>zpool status > > pool: st25 > > state: ONLINE > > scan: none requested > > config: > > > > NAME STATE READ WRITE CKSUM > > st25 ONLINE 0 0 0 > > mirror-0 ONLINE 0 0 0 > > ada0 ONLINE 0 0 0 > > ada1 ONLINE 0 0 0 > > mirror-1 ONLINE 0 0 0 > > ada2 ONLINE 0 0 0 > > ada3 ONLINE 0 0 0 > > mirror-2 ONLINE 0 0 0 > > ada4 ONLINE 0 0 0 > > ada5 ONLINE 0 0 0 > > mirror-3 ONLINE 0 0 0 > > ada6 ONLINE 0 0 0 > > ada7 ONLINE 0 0 0 > > > > errors: No known data errors > > > > They haven't separate boot device and have from 7 to 30+ Tb of data that I > > can't backup anywhere for re-install server with disk partitioning. Some > > time ago I managed to get aroud this reverting 2 commits ( r342151 + don't > > remember revision) but then there where some other improvements after which > > I lost the opportunity to revert this commits > > > > -- > > wbr, Sergey > > > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > -- wbr, Sergey From owner-freebsd-stable@freebsd.org Thu Oct 22 20:42:41 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F34C042AF80; Thu, 22 Oct 2020 20:42:40 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: from mail-ej1-x644.google.com (mail-ej1-x644.google.com [IPv6:2a00:1450:4864:20::644]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CHK742Qf1z4V7h; Thu, 22 Oct 2020 20:42:40 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: by mail-ej1-x644.google.com with SMTP id t25so4265987ejd.13; Thu, 22 Oct 2020 13:42:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=uKN8zarZTZvUvNPtw6xVbtJKFFvlrOO9k0vFAmyW5wc=; b=otlfVvrd6BAAmbXIC8hm/TS8nAyPHyb4lhEmHsms8rE2fxXjtLlpUXV3DP4RU9e+A2 AFp5wjM0itJbwl/EaapWDaR1Wpm1ccvzXsRbbRGeQI7sXvttkRpQcmvPq5n2AJ9X18OJ tpKTmZbuOjdVc0nAoSmqlB7a795jT7bQKhs68TWgFNpWbxCXpi5yKRDajyrveLdWuNgN kDA+iOaDNxmkT4UEkfGeK/DcdeTeuCARo1GVlsL6aroVaq7m9Ebm7XwwXsxm8FDw+b/I IYsyj9ziQDxYin+hxm7aAAkaRoflPJKd2TykfQxuivD5KdAvC9ojd6SW27Z8S4BcoRcq 9GKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=uKN8zarZTZvUvNPtw6xVbtJKFFvlrOO9k0vFAmyW5wc=; b=oqq7U9a8+OIg88F2X0csHCRqFpuDxhmLgmErrcY84vcVvrPTPZbnghczASMBGCAumJ YoWjdHFA3QM2K47pNr03fCtyO/VKbDsqYQS2OcKv+DkpkVRFsDzJBhvMqbZ0DMf1n6vp SQ6IotzB94prg6/vRyG+sxRBaTrTugGuDuoFmWCQ494ZpaC47iQmaSGMayUPTTesbxEw Sk74MyW3vVnD7t2hT/TcEUnQ3hJcCeZvaYw7lGTnUq17g/REp4KOmtOcQPhcGUoXuEiL 4QO0QpDm1dXjNqY9gvX4oL5tMmCotA9LlcG/bwQGniM17VB8t9V8Utgk+IlhwP5nZvIA DUQQ== X-Gm-Message-State: AOAM530g1igVD0q6mE8E/6c4WFc5LhfybgBGLH7I7jDdDqHzmovy7/ky 92Ck0S1+8z7W421TLHhdWuUVKqVc6Q+6PrFGakc= X-Google-Smtp-Source: ABdhPJyLSvmXMJgMBEQEqYPSDpZC6O7OWJd124mLknK+btHp88RQutDgWiFkmoDOKDxxvEaVrGu29JJNvBLAlFM7Gvg= X-Received: by 2002:a17:906:3799:: with SMTP id n25mr4016631ejc.6.1603399358861; Thu, 22 Oct 2020 13:42:38 -0700 (PDT) MIME-Version: 1.0 References: <33381216-498a-51ff-839c-da772c43b60e@FreeBSD.org> <20201022203752.36fb6f3f@laptop.domain> In-Reply-To: From: Cassiano Peixoto Date: Thu, 22 Oct 2020 17:42:26 -0300 Message-ID: Subject: Re: Zpool doesn't boot anymore after FreeBSD 12.1 To: Toomas Soome Cc: "Sergey V. Dyatko" , freebsd-current@freebsd.org, freebsd-stable@freebsd.org X-Rspamd-Queue-Id: 4CHK742Qf1z4V7h X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=otlfVvrd; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of peixotocassiano@gmail.com designates 2a00:1450:4864:20::644 as permitted sender) smtp.mailfrom=peixotocassiano@gmail.com X-Spamd-Result: default: False [-1.86 / 15.00]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-stable]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.40)[-0.402]; FREEMAIL_TO(0.00)[me.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.94)[-0.942]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.02)[-1.016]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::644:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; SUSPICIOUS_RECIPS(1.50)[]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2020 20:42:41 -0000 Ok, thanks. Please keep me posted. On Thu, Oct 22, 2020 at 5:37 PM Toomas Soome wrote: > > > On 22. Oct 2020, at 23:29, Cassiano Peixoto > wrote: > > > Hi Toomas, > > Thank you for this patch. Can I know when it will be MFCed? I'd like to > try. > > > > MFC will take a bit because I need to bring it together with asize > updates, so we would need to do a bit of cherry-picking and testing. > > rgds, > toomas > > > > > Hi! > > Please try 366951:) I think, it should get things better for you. > > rgds, > toomas > > On Thu, Oct 22, 2020 at 3:04 PM Cassiano Peixoto < > peixotocassiano@gmail.com> wrote: > >> Sergey, >> >> I agree with you. The best thing is to revert the commit on zfsloader. >> Many people didn't realize this issue yet, but they will run into a big >> problem soon. >> >> On Thu, Oct 22, 2020 at 2:41 PM Sergey V. Dyatko >> wrote: >> >>> On Thu, 22 Oct 2020 16:42:16 +0300 >>> Andriy Gapon wrote: >>> >>> > On 22/10/2020 16:39, Cassiano Peixoto wrote: >>> > > Hi Andriy, >>> > > >>> > > I've just tried copying my zfsloader from 11.2-STABLE (R350026) to >>> FreeBSD >>> > > 12.1 and 12.2 (STABLE) and fixed the issue. >>> > > >>> > > I also tried to use zfsloader of 11.3 but didn't work and the same >>> issue >>> > > happened. >>> > > >>> > > So it seems that something has changed on zfsloader after 11.2 that >>> brings >>> > > this issue. >>> > > >>> > > My question is: Should it be expected or is it a bug to be fixed? >>> > > >>> > >>> > In my opinion it's a bug. >>> > zfsloader should not require that disks must be partitioned. >>> > >>> >>> +1 >>> That's why I have terribly outdated 13-CURRENT on bunch of servers and >>> can't >>> update them. >>> >>> Most of them look like this: >>> [tiger@st25]:~>zpool status >>> pool: st25 >>> state: ONLINE >>> scan: none requested >>> config: >>> >>> NAME STATE READ WRITE CKSUM >>> st25 ONLINE 0 0 0 >>> mirror-0 ONLINE 0 0 0 >>> ada0 ONLINE 0 0 0 >>> ada1 ONLINE 0 0 0 >>> mirror-1 ONLINE 0 0 0 >>> ada2 ONLINE 0 0 0 >>> ada3 ONLINE 0 0 0 >>> mirror-2 ONLINE 0 0 0 >>> ada4 ONLINE 0 0 0 >>> ada5 ONLINE 0 0 0 >>> mirror-3 ONLINE 0 0 0 >>> ada6 ONLINE 0 0 0 >>> ada7 ONLINE 0 0 0 >>> >>> errors: No known data errors >>> >>> They haven't separate boot device and have from 7 to 30+ Tb of data that >>> I >>> can't backup anywhere for re-install server with disk partitioning. Some >>> time >>> ago I managed to get aroud this reverting 2 commits ( r342151 + don't >>> remember revision) but then there where some other improvements after >>> which I >>> lost the opportunity to revert this commits >>> >>> -- >>> wbr, Sergey >>> >>> _______________________________________________ >>> freebsd-stable@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >>> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org >>> " >>> >> >