From owner-freebsd-current@freebsd.org Fri May 14 12:52:21 2021 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3600F64695F; Fri, 14 May 2021 12:52:21 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FhT2C6pwwz3jsj; Fri, 14 May 2021 12:52:18 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id 14ECqB8W081136; Fri, 14 May 2021 05:52:11 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id 14ECqA0h081135; Fri, 14 May 2021 05:52:10 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <202105141252.14ECqA0h081135@gndrsh.dnsmgr.net> Subject: Re: FYI for aarch64 main [14] running a mid March version: I ended up with [usb{usbus2}] stuck at (near) 100% cpu In-Reply-To: To: Mark Millard Date: Fri, 14 May 2021 05:52:10 -0700 (PDT) CC: freebsd-arm , freebsd-current X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 4FhT2C6pwwz3jsj X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of freebsd-rwg@gndrsh.dnsmgr.net has no SPF policy when checking 69.59.192.140) smtp.mailfrom=freebsd-rwg@gndrsh.dnsmgr.net X-Spamd-Result: default: False [-2.08 / 15.00]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[dnsmgr.net]; RBL_DBL_DONT_QUERY_IPS(0.00)[69.59.192.140:from]; AUTH_NA(1.00)[]; SPAMHAUS_ZRD(0.00)[69.59.192.140:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.98)[-0.976]; R_SPF_NA(0.00)[no SPF record]; FREEMAIL_TO(0.00)[yahoo.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm,freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 May 2021 12:52:21 -0000 > Note: The context was using a non-debug main build > from mid-2021-Mar. (More details identified > later.) > > The issue happend while attempting a: > > # zfs send -R zpold@for-copy | zfs recv -Fdv zpnew > > where the drives involved in the command were: > > zpold: a USB3 SSD, using /dev/da0p3 > zpnew: an 480 GiByte Optane in the PCIe slot, using /dev/nda0p3 > > with: > > # gpart show -pl > => 40 468862048 da0 GPT (224G) > 40 532480 da0p1 4C8GCA72EFI (260M) > 532520 2008 - free - (1.0M) > 534528 29360128 da0p2 4C8GCA72swp14 (14G) > 29894656 4194304 - free - (2.0G) > 34088960 33554432 da0p4 4C8GCA72swp16 (16G) > 67643392 401217536 da0p3 4C8GCA72zfs (191G) > 468860928 1160 - free - (580K) > > => 40 2000409184 ada0 GPT (954G) > 40 409600 ada0p1 (null) (200M) > 409640 1740636160 ada0p2 FBSDmacchroot (830G) > 1741045800 58720256 ada0p3 FBSDmacchswp0 (28G) > 1799766056 176160768 ada0p4 FBSDmacchswp1 (84G) > 1975926824 24482400 - free - (12G) > > => 40 937703008 nda0 GPT (447G) > 40 532480 nda0p1 CA72opt0EFI (260M) > 532520 2008 - free - (1.0M) > 534528 117440512 nda0p2 CA72opt0swp56 (56G) > 117975040 16777216 - free - (8.0G) > 134752256 134217728 nda0p4 CA72opt0swp64 (64G) > 268969984 668731392 nda0p3 CA72opt0zfs (319G) > 937701376 1672 - free - (836K) > > The system running was that on /dev/ada0p2 (FBSDmacchroot, > which is UFS instead of ZFS). > > The [usb{usbus2}] process eventually got stuck-busy, no > more I/O: > > CPU 0: 0.0% user, 0.0% nice, 100% system, 0.0% interrupt, 0.0% idle > CPU 1: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle > CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle > CPU 3: 0.4% user, 0.0% nice, 0.0% system, 0.0% interrupt, 99.6% idle > > PID USERNAME PRI NICE SIZE RES STATE C TIME CPU COMMAND > 15 root -72 - 0B 262144B CPU0 0 8:51 99.95% [usb{usbus2}] > > 1295 root -8 0 20108Ki 8092Ki q->bq_ 2 0:04 0.00% zfs recv -Fdv zpnew{receive_writer_thre} > 1295 root 48 0 20108Ki 8092Ki piperd 2 0:22 0.00% zfs recv -Fdv zpnew{zfs} > 1294 root -8 0 17544Ki 7740Ki q->bq_ 2 0:01 0.00% zfs send -R zpold@for-copy{send_reader_thread} > 1294 root -8 0 17544Ki 7740Ki q->bq_ 0 0:00 0.00% zfs send -R zpold@for-copy{send_merge_thread} > 1294 root -8 0 17544Ki 7740Ki hdr->b 2 0:00 0.00% zfs send -R zpold@for-copy{send_traverse_threa} > 1294 root 52 0 17544Ki 7740Ki range- 3 0:20 0.00% zfs send -R zpold@for-copy{zfs} > > 1036 root -8 - 0B 1488Ki t->zth 0 0:00 0.00% [zfskern{z_checkpoint_discar}] > 1036 root -8 - 0B 1488Ki t->zth 1 0:00 0.00% [zfskern{z_livelist_condense}] > 1036 root -8 - 0B 1488Ki t->zth 2 0:00 0.00% [zfskern{z_livelist_destroy}] > 1036 root -8 - 0B 1488Ki t->zth 1 0:00 0.00% [zfskern{z_indirect_condense}] > 1036 root -8 - 0B 1488Ki mmp->m 3 0:00 0.00% [zfskern{mmp_thread_enter}] > 1036 root -8 - 0B 1488Ki tx->tx 1 0:00 0.00% [zfskern{txg_thread_enter}] > 1036 root -8 - 0B 1488Ki tx->tx 2 0:00 0.00% [zfskern{txg_thread_enter}] > > I was unable to ^c or ^z the process where I > typed the command. I eventually stopped the > system with "shutdown -p now" from a ssh > session (that had already been in place). Should this occur again before doing the shutdown run a zpool status & I have gotten in this state when the recv pool was a usb device and for some reason it had a timeout and gone offline. The clue this occured are in dmesg, and zpool status. Unplug/plug the USB device, check dmesg that it came online, and do a zpool clear. > > When I retried after rebooting and scrubbing (no > problems found), the problem did not repeat. > > I do not have more information nor a way to repeat > the problem on demand, unfortunately. > > Details of the vintage of the system software and > such: > > # ~/fbsd-based-on-what-freebsd-main.sh > FreeBSD FBSDmacch 14.0-CURRENT FreeBSD 14.0-CURRENT mm-src-n245445-def0058cc690 GENERIC-NODBG arm64 aarch64 1400005 1400005 > def0058cc690 (HEAD -> mm-src) mm-src snapshot for mm's patched build in git context. > merge-base: 7381bbee29df959e88ec59866cf2878263e7f3b2 > merge-base: CommitDate: 2021-03-12 20:29:42 +0000 > 7381bbee29df (freebsd/main, freebsd/HEAD, pure-src, main) cam: Run all XPT_ASYNC ccbs in a dedicated thread > n245444 (--first-parent --count for merge-base) > > The system was a MACCHIATObin Double Shot. > > === > Mark Millard > marklmi at yahoo.com > ( dsl-only.net went > away in early 2018-Mar) > > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > -- Rod Grimes rgrimes@freebsd.org