From owner-freebsd-arm@freebsd.org Sat Apr 17 09:38:14 2021 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 330CD5F6E02 for ; Sat, 17 Apr 2021 09:38:14 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FMp0j0QpYz4W2p for ; Sat, 17 Apr 2021 09:38:12 +0000 (UTC) (envelope-from ronald-lists@klop.ws) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=klop.ws; s=mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version: Date:Message-ID:From:References:To:Subject:Sender:Reply-To:Cc:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=NbolpYjRAVJpYuGPRi4Gku8D2B4dEidAk4E70qetiCQ=; b=BlC0k8LmlhABACFhuDJgP3KUAV XJzPyblr94SM6oOy1q0cvOQWaUN5K33rGdqmJ7BSKcYlsTcqfLT8XfCXcNNR14B6JgUKAN+gRXs5X cM0doL1i9l1JCJTyQeKgUER77aZH/g8MYq+Q9vNapEajsFS2I+cqoOOcgI6f8YQsRdGY=; Subject: Re: RPi 4B USB 3 support appears to still be broken in 13.0-RELEASE To: freebsd-arm@freebsd.org References: From: Ronald Klop Message-ID: <187de31a-e96f-aa98-23a4-e0fb63de5671@klop.ws> Date: Sat, 17 Apr 2021 11:38:05 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.9.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Authenticated-As-Hash: 398f5522cb258ce43cb679602f8cfe8b62a256d1 X-Virus-Scanned: by clamav at smarthost1.greenhost.nl X-Spam-Level: / X-Spam-Score: -0.4 X-Spam-Status: No, score=-0.4 required=5.0 tests=ALL_TRUSTED, BAYES_50, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, NICE_REPLY_A autolearn=disabled version=3.4.2 X-Scan-Signature: 66f4fda096222dd2b2010deb1ce817c5 X-Rspamd-Queue-Id: 4FMp0j0QpYz4W2p X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=klop.ws header.s=mail header.b=BlC0k8Lm; dmarc=pass (policy=none) header.from=klop.ws; spf=pass (mx1.freebsd.org: domain of ronald-lists@klop.ws designates 195.190.28.88 as permitted sender) smtp.mailfrom=ronald-lists@klop.ws X-Spamd-Result: default: False [-1.96 / 15.00]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[klop.ws:s=mail]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.190.28.64/27]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; RCPT_COUNT_ONE(0.00)[1]; DKIM_TRACE(0.00)[klop.ws:+]; DMARC_POLICY_ALLOW(-0.50)[klop.ws,none]; RCVD_IN_DNSWL_NONE(0.00)[195.190.28.88:from]; NEURAL_HAM_SHORT(-0.96)[-0.964]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RWL_MAILSPIKE_VERYGOOD(0.00)[195.190.28.88:from]; ASN(0.00)[asn:47172, ipnet:195.190.28.0/24, country:NL]; MID_RHS_MATCH_FROM(0.00)[]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Porting FreeBSD to ARM processors List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Apr 2021 09:38:14 -0000 On 4/17/21 1:00 AM, Robert Clausecker wrote: > Greetings! > > Last time I experimented with ZFS on the RPi 4B, I noticed that > there is a strange problem when attaching the zpool via USB 3 as > opposed to USB 2. When doing that, mounting root fails with > IO errors like these: > > (da0:umass-sim0:0:0:0): READ(10). CDB: 28 00 03 c1 b9 65 00 00 07 00 > (da0:umass-sim0:0:0:0): CAM status: CCB request completed with an error > (da0:umass-sim0:0:0:0): Retrying command, 3 more tries remain > (da0:umass-sim0:0:0:0): READ(10). CDB: 28 00 03 c1 b9 65 00 00 07 00 > (da0:umass-sim0:0:0:0): CAM status: CCB request completed with an error > (da0:umass-sim0:0:0:0): Retrying command, 2 more tries remain > > Attaching the boot disk through USB 2 instead works. Likewise, > using USB 3 with a UFS root file system works (and in fact ran fine > in a development system for months). I do not understand this. > > I had previously reported this issue as PR 249520: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=249520 > > There's some stuff about UEFI booting in there which you can ignore. > The same problem also appears when booting via U-Boot. > > Now what surprises me is that this issue still occurs with > FreeBSD 13.0-RELEASE. So whatever fixes had been performed > did not seem to address the underlying problem at all. > > Is there any workaround or solution (except for ditching root > on ZFS which would be rather painful for my use case?) > > Yours, > Robert Clausecker > Weird that UFS works, but ZFS doesn't. This is really on the same disk? Just checking. I can only add a datapoint that I'm running 13.0 on ZFS on USB3 on RPI4-8GB without problems. Could it be a partitioning difference that you are crossing 4K-sector boundaries or something else that amplifies the traffic when using ZFS? Or bad sectors which are nog used with UFS? Another possibility with your hardware: https://twitter.com/bsdimp/status/1383165361734832132?s=20 13.0 increased maxphys which is the size of the data transfers send to the disk. Can you try kern.maxphys=131072 in loader.conf? [root@rpi4 ~]# usbconfig ugen0.1: <0x1106 XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA) ugen0.2: at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (100mA) ugen0.3: at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA) <--- Regards, Ronald.