From owner-freebsd-fs@FreeBSD.ORG Thu Jun 14 12:10:56 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (unknown [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E4957106567A for ; Thu, 14 Jun 2012 12:10:56 +0000 (UTC) (envelope-from devgs@ukr.net) Received: from ffe5.ukr.net (ffe5.ukr.net [195.214.192.21]) by mx1.freebsd.org (Postfix) with ESMTP id 72FBC8FC0C for ; Thu, 14 Jun 2012 12:10:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Date:Message-Id:From:To:References:In-Reply-To:Subject:Cc:Content-Type:Content-Transfer-Encoding:MIME-Version; bh=fKfnF6OdK7v+hs4ncOQfcSoPQsIwf/GL+hy+2Q9mHv4=; b=TbyuaCrxYdgzm/+espkS5T1rqSyJMWluIIYgE2T0Y+A8nZwJa8lx5GgxOyy+NsKDzz/AVxjFUwOonQOI7h/qLpPntgyFkOwDcee0Gg2PGWLFDxl/UWofwUTCFDqdajtRnWS6jFuhJYZ/Vzkq1A0CXmfG8Zpu3/LfYs6hRBZ35/c=; Received: from mail by ffe5.ukr.net with local ID 1Sf8ZZ-0005UR-59 ; Thu, 14 Jun 2012 14:50:45 +0300 MIME-Version: 1.0 In-Reply-To: <893489718.1762311.1339673556220.JavaMail.root@erie.cs.uoguelph.ca> References: <893489718.1762311.1339673556220.JavaMail.root@erie.cs.uoguelph.ca> To: "Rick Macklem" From: "Pavlo" X-Mailer: freemail.ukr.net 4.0 X-Originating-Ip: [212.42.94.154] Message-Id: <14119.1339674645.325291171772956672@ffe5.ukr.net> X-Browser: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:13.0) Gecko/20100101 Firefox/13.0 Date: Thu, 14 Jun 2012 14:50:45 +0300 Content-Type: text/plain; charset="windows-1251" Content-Transfer-Encoding: binary Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: mmap() incoherency on hi I/O load (FS is zfs) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jun 2012 12:10:57 -0000 > Pavlo wrote: > There's a case when some parts of files that are mapped and then > modified getting corrupted. By corrupted I mean some data is ok (one > that > was written using write()/pwrite()) but some looks like it never > existed. > Like it was some time in buffers, when several processes > simultaneously > (of course access was synchronised) used shared pages and reported > it's > existence. But after time pass they (processes) screamed that it is > now > lost. Only part of data written with pwrite() was there. Everything > that > was written via mmap() is zero. > > So as I said it occurs on hi I/O busyness. When in background 4+ > processes do indexing of huge ammount of data. Also I want to note, it > never occurred in the life of our project while we used mmap() under > same I/O stress conditions when mapping was done for a whole file of > just > a part(header) starting from a beginning of a file. First time we used > mapping of individual pages, just to save RAM, and this popped up. > > Solution for this problem is msync() before any munmap(). But man > says: > > The msync() system call is usually not needed since BSD implements a > coherent file system buffer cache. However, it may be used to > associate > dirty VM pages with file system buffers and thus cause them to be > flushed > to physical media sooner rather than later. > > Any thoughts? Thanks. > With a recent kernel from head, I am seeing dirty mmap'd pages being written quite late for the NFSv4 client. Even after the NFS client VOP_RECLAIM() has been called, it seems. I didn't observe this behaviour in a kernel from head in March. (I don't know enough about the vm/mmap area to know if this is correct behaviour or not?) I thought I'd mention this, since you didn't say how recent a kernel you were running and thought it might be caused by the same change? Sorry I can't help more, rick > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > Thanks for reply, Rick! Yes, we have pretty old kernel: # uname -a FreeBSD mpop-zebra-k1.ukr.net 8.2-STABLE FreeBSD 8.2-STABLE #9: Wed Jan 25 11:28:55 EET 2012 I just posted my observation here to point out possible problem that could still exist.