From nobody Sat Feb 19 17:02:40 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id CAAFE19C7EE2 for ; Sat, 19 Feb 2022 17:03:02 +0000 (UTC) (envelope-from mike@karels.net) Received: from mail.karels.net (mail.karels.net [216.160.39.52]) by mx1.freebsd.org (Postfix) with ESMTP id 4K1FHn3Bbgz3ChQ; Sat, 19 Feb 2022 17:03:01 +0000 (UTC) (envelope-from mike@karels.net) Received: from mail.karels.net (localhost [127.0.0.1]) by mail.karels.net (8.16.1/8.16.1) with ESMTP id 21JH2eLS077912; Sat, 19 Feb 2022 11:02:41 -0600 (CST) (envelope-from mike@karels.net) Received: from [10.0.2.130] ([10.0.1.1]) by mail.karels.net with ESMTPSA id fLCfFLAiEWJWMAEA4+wvSQ (envelope-from ); Sat, 19 Feb 2022 11:02:40 -0600 From: Mike Karels To: Tomoaki AOKI Cc: Alexander Motin , "Chen, Alvin W" , freebsd-current@freebsd.org Subject: Re: [Intel AlderLake] Read&Write files to FAT32 or UFS partition cause data corrupt due to P-Core&E-Core Date: Sat, 19 Feb 2022 11:02:40 -0600 X-Mailer: MailMate (1.14r5818) Message-ID: <7A743668-B5AA-4679-9F56-9A6220CBBC14@karels.net> In-Reply-To: <20220219115534.7db1b9f199c10894e4280b33@dec.sakura.ne.jp> References: <5fd2a34e-1135-4237-a028-d4566ff65c69@FreeBSD.org> <20220219115534.7db1b9f199c10894e4280b33@dec.sakura.ne.jp> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4K1FHn3Bbgz3ChQ X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@karels.net designates 216.160.39.52 as permitted sender) smtp.mailfrom=mike@karels.net X-Spamd-Result: default: False [-3.01 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[mike]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:216.160.39.52]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[karels.net]; NEURAL_HAM_LONG(-0.81)[-0.814]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-0.998]; MLMMJ_DEST(0.00)[freebsd-current]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:209, ipnet:216.160.36.0/22, country:US]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On 18 Feb 2022, at 20:55, Tomoaki AOKI wrote: > Just a thought, but can it be the reason with timing (e.g., rendezvous > within (i)threads, hardware controlls without using hardware timer) > problem? > > On FreeBSD, IIUC, multi processor (multi core) implementation assumes > SMP (differs only clock speed) and end up with difference of > performance at same clock speed within P-core and E-core, possibly. Another possibility is that the system is confused by having hyperthreadi= ng on the P cores but not the E cores. > BTW, how aarch64 guys implement big.Little support to avoid such a case= ? > Or are they simply disable all Little cores and use big only? Are there supported arm64 systems with asymmetric processors yet? Mike > On Fri, 18 Feb 2022 15:36:27 -0500 > Alexander Motin wrote: > >> This looks pretty weird to me, but I don't think it is specific to the= >> FAT32. Just today I've first noticed that booting TrueNAS 12.0-U8 >> (http://download.freenas.org/12.0/STABLE/U8/x64/TrueNAS-12.0-U8.iso) >> (based on FreeBSD 12.2 with many backports) from NVMe SSD (I don't >> insist on NVMe so far) and ZFS almost never completes successfully, >> ending up in hangs or random stack corruption panics in ZFS threads as= >> soon as at least one E core is enabled of my i7-12700K. Disabling all= E >> cores fixes the problem. Updated to TrueNAS 13.0-BETA1 (based on >> FreeBSD 13.0-STABLE from few weeks ago) it does not demonstrate the >> problem any more. The same TrueNAS 12.0-U8 kernel booted from NFS doe= s >> not seem to demonstrate the problem with ZFS mounting, but I haven't >> stressed it much so far. >> >> There are seem to be dragons somewhere... >> >> On 15.02.2022 22:29, Chen, Alvin W wrote: >>> Hi Guys, >>> Any updates to support Intel P-core + E-core? >>> I have filed a bug: PR 261169 >>> , but no= updates. >>> Does anybody know the progress? >>> >>> For Intel Adler Lake P core + E core processor (i7-12700T), copying >>> files to FAT32 partition, the file corrupted (50%), but ZFS is fine. >>> After disabling E core in the code by restrict the max cpu number, th= is >>> issue is gone. And No E core processor has no such issue, like i7-124= 00. >>> >>> HW ENV: >>> CPU: Intel AlderLake 12th Gen i7-12700T >>> Disk: NVME SSD >>> >>> There are 3 methods to reproduce this issue: >>> 1. Make FreeBSD 13 USB disk installer, install FreeBSD with UFS, and >>> select install source and ports, the txz package checking will be fai= led. >>> >>> 2. Boot to shell by USB disk installer, and mount a FAT32 partition (= on >>> SSD), and copy a 300MB file to the FAT32, compare the sha256 checksum= s >>> for the source file and the dst file, the checksum are different (50%= ). >>> Or if there is a 300MB file in FAT32 partition, mount the partition, = and >>> for the first time check the sha256 value by running 'sha256 file.tgz= ', >>> the checksum is wrong, but the second time, the checksum is correct. >>> >>> 3. Install FreeBSD 13 with ZFS, and it can work well. And boot into >>> FreeBSD, disable swap, and format the SWAP partition to FAT32. Do the= >>> testing as above. >>> >>> Regards, >>> >>> Alvin Chen >>> >>> Dell | Comercial Client Group >>> >>> office +86-10-82862506, fax +86-10-82861554, Dell Lync 8672506 >>> weike_chen@dell.com >>> >>> Internal Use - Confidential >>> >> >> -- = >> Alexander Motin >> > > > -- = > =E9=9D=92=E6=9C=A8 =E7=9F=A5=E6=98=8E [Tomoaki AOKI]