From owner-freebsd-hackers@freebsd.org Sun May 22 23:12:12 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4AE3DB3B8FA for ; Sun, 22 May 2016 23:12:12 +0000 (UTC) (envelope-from joerg@bec.de) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [IPv6:2001:4b98:c:538::197]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 12F8D16C7 for ; Sun, 22 May 2016 23:12:12 +0000 (UTC) (envelope-from joerg@bec.de) Received: from mfilter33-d.gandi.net (mfilter33-d.gandi.net [217.70.178.164]) by relay5-d.mail.gandi.net (Postfix) with ESMTP id C320141C088 for ; Mon, 23 May 2016 01:12:10 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at mfilter33-d.gandi.net Received: from relay5-d.mail.gandi.net ([IPv6:::ffff:217.70.183.197]) by mfilter33-d.gandi.net (mfilter33-d.gandi.net [::ffff:10.0.15.180]) (amavisd-new, port 10024) with ESMTP id gvN-kEVR4Yt8 for ; Mon, 23 May 2016 01:12:09 +0200 (CEST) X-Originating-IP: 108.61.101.156 Received: from britannica.bec.de (unknown [108.61.101.156]) (Authenticated sender: joerg@bec.de) by relay5-d.mail.gandi.net (Postfix) with ESMTPSA id 6074E41C086 for ; Mon, 23 May 2016 01:12:07 +0200 (CEST) Date: Mon, 23 May 2016 01:12:03 +0200 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org Subject: Re: read(2) and thus bsdiff is limited to 2^31 bytes Message-ID: <20160522231203.GB25503@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org References: <20160522225414.GB24398@britannica.bec.de> <154dab43060.11208cdfd132112.2616144627831899155@nextbsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <154dab43060.11208cdfd132112.2616144627831899155@nextbsd.org> User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 May 2016 23:12:12 -0000 On Sun, May 22, 2016 at 04:02:02PM -0700, Matthew Macy wrote: > > > > ---- On Sun, 22 May 2016 15:54:14 -0700 Joerg Sonnenberger wrote ---- > > On Sun, May 22, 2016 at 10:54:30PM +0200, Dirk Engling wrote: > > > When trying to bsdiff two DVD images, I noticed it failing due to > > > read(2) returning EINVAL to the tool. man 2 read says, this would only > > > happen for a negative value for fildes, which clearly was not true. > > > > I would classify that as implementation bug. It seems perfectly sensible > > to turn overly large requests into a short read/write, even for blocking > > files. But erroring out seems to be quite wrong to me. > > > > read(2) takes a size_t so this is clearly an internal bug where it's an int and treating it as a negative value. Not exactly. The reason for cutting it off are many fold. Using int in the kernel is one argument. The requirement for locking the IO range for concurrent read/write operations from other threads is a bigger argument. Joerg