From owner-freebsd-hackers@freebsd.org Fri Apr 26 17:20:22 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 614C11599408 for ; Fri, 26 Apr 2019 17:20:22 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lj1-f177.google.com (mail-lj1-f177.google.com [209.85.208.177]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 53245757D7 for ; Fri, 26 Apr 2019 17:20:21 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lj1-f177.google.com with SMTP id i186so3597606lji.9 for ; Fri, 26 Apr 2019 10:20:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:cc; bh=KixhjoKdtgMObyZ0xFhmHmDTALxLJwEvhYwKYSw76aE=; b=CYsaXpt9UGmDQpAYk/C0DlaqN0ccGTErb9TId/3iA1KjIutMC3LKJH+WYQbfJGWoCu HCu+sx8avtKpqQR6nHLI3GXUK90QjP0a2kcn6QIzHtBFu/vFVfxE8XVgGSo/p1TKxBP0 TqZW4hCXYDJkq/5iGO5+GNDo1T2MEZcfp4xRA6L040tVOsmvvJ0yfVr6doX3XcEsLpEH Wi56NEOMj0PXDVLL1PzL8LTmdnDFlF3MAc8SU+j4yCTnXbuq7exeHNQOgW2MMNMoFt3t A8lStTQNXVXnJrTcc791FkqlnVKSjO8L+0C/gYCzl+HtneTbxDE9+MxeoF0j7pbcWY+L ISSw== X-Gm-Message-State: APjAAAUU0mKJwVn0isSVZUPJJFUQ5oi8bkaPkmzsHTIqrw9ocwvwSXGv QiY5Ro2pGzLItx9azHvWpooKZwIt6OUO/zsnSute0vHu7MU= X-Google-Smtp-Source: APXvYqw0vBlm8ohpzz1BcnetW5Cbf3CfYuDRsraCtSSptRKHQtFq+tpREowzRAQBc/zo1/yT7vFFkYataVgyntWLqMQ= X-Received: by 2002:a2e:9f14:: with SMTP id u20mr10628223ljk.108.1556299213356; Fri, 26 Apr 2019 10:20:13 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Fri, 26 Apr 2019 11:20:01 -0600 Message-ID: Subject: Re: buf(9) woes: when does bcopy do nothing at all? Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 53245757D7 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.208.177 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-1.29 / 15.00]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.94)[-0.942,0]; MISSING_TO(2.00)[]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; SUBJECT_ENDS_QUESTION(1.00)[]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.993,0]; TO_DOM_EQ_FROM_DOM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.99)[-0.986,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-1.36)[ip: (-0.61), ipnet: 209.85.128.0/17(-3.87), asn: 15169(-2.26), country: US(-0.06)]; RCVD_IN_DNSWL_NONE(0.00)[177.208.85.209.list.dnswl.org : 127.0.5.0]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2019 17:20:22 -0000 On Thu, Apr 25, 2019 at 9:31 PM Alan Somers wrote: > > How is it possible that bcopy() doesn't affect its output array at > all? While investigating a data corruption issue in fuse, I narrowed > the problem down to a bcopy operation that apparently has no affect. > The code in question is: > > bcopy(cp, iov->iov_base, cnt); > r = memcmp(cp, iov->iov_base, cnt); > if (r) > printf("uiomove: miscompare\n"); > > Rationally, I would expect that line never to be printed. But it > does. The destination is always all zeros, even though the source is > not. I can only guess that there's something wrong about the way that > I I'm using buf(9), because the output is part of a buffer allocated > by bread(9). I've been able to rule out: > > 1) Race conditions. The bug is 100% reproducible, and doubling the > bcopy or changing the timing in other ways has no effect. > 2) Unmapped buffer. I verified that the buf is not unmapped_buf. > 3) Overlapping src and dst > 4) Duplicated pages. I verified that each of the buf's pages has a > unique physical address > 5) Bad RAM. My machine passes memtest86, and anyway the failure is > too specific and reproducible to be caused by bad hardware. > > What could I be missing? Do I need to do something to prepare the buf > before I can use it? The code that allocates the buffer is here: > https://svnweb.freebsd.org/base/projects/fuse2/sys/fs/fuse/fuse_io.c?view=markup#l240 > > -Alan To answer phk's questions, I checked that src and dst don't overlap, and the kernel's bcopy is actually a wrapper around memmove. To answer hps's question, this is on amd64, in a bhyve VM. I solved the problem - part of it, anyway. The user-visible problem that originally led me down this rabbit-hole was an apparent cache invalidation failure during writes on fusefs. That turned out to be caused by an off-by-one error that I just fixed in r346756. However, the miscompares remain. Could those pages be mapped differently for reading than for writing? I don't know. At this point, I'm not going to put much more effort into investigating the problem; I've wasted too much time already. -Alan