From owner-freebsd-hackers@freebsd.org Mon May 23 12:21:36 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2B755B459B7 for ; Mon, 23 May 2016 12:21:36 +0000 (UTC) (envelope-from joerg@bec.de) Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [IPv6:2001:4b98:c:538::196]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EBEA817D6 for ; Mon, 23 May 2016 12:21:35 +0000 (UTC) (envelope-from joerg@bec.de) Received: from britannica.bec.de (p20030057E21176048C47C6DE7E800516.dip0.t-ipconnect.de [IPv6:2003:57:e211:7604:8c47:c6de:7e80:516]) (Authenticated sender: joerg@bec.de) by relay4-d.mail.gandi.net (Postfix) with ESMTPSA id 6A8051720BD; Mon, 23 May 2016 14:21:33 +0200 (CEST) Date: Mon, 23 May 2016 14:21:31 +0200 From: Joerg Sonnenberger To: Matthew Macy Cc: Joerg Sonnenberger , "" Subject: Re: read(2) and thus bsdiff is limited to 2^31 bytes Message-ID: <20160523122131.GC8747@britannica.bec.de> Mail-Followup-To: Matthew Macy , Joerg Sonnenberger , "" References: <20160522225414.GB24398@britannica.bec.de> <154dab43060.11208cdfd132112.2616144627831899155@nextbsd.org> <20160522231203.GB25503@britannica.bec.de> <154db353935.dd5e87c1133922.4370692881788049491@nextbsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <154db353935.dd5e87c1133922.4370692881788049491@nextbsd.org> User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 May 2016 12:21:36 -0000 On Sun, May 22, 2016 at 06:22:58PM -0700, Matthew Macy wrote: > > > > ---- On Sun, 22 May 2016 16:12:03 -0700 Joerg Sonnenberger wrote ---- > > On Sun, May 22, 2016 at 04:02:02PM -0700, Matthew Macy wrote: > > > > > > > > > > > > ---- On Sun, 22 May 2016 15:54:14 -0700 Joerg Sonnenberger wrote ---- > > > > On Sun, May 22, 2016 at 10:54:30PM +0200, Dirk Engling wrote: > > > > > When trying to bsdiff two DVD images, I noticed it failing due to > > > > > read(2) returning EINVAL to the tool. man 2 read says, this would only > > > > > happen for a negative value for fildes, which clearly was not true. > > > > > > > > I would classify that as implementation bug. It seems perfectly sensible > > > > to turn overly large requests into a short read/write, even for blocking > > > > files. But erroring out seems to be quite wrong to me. > > > > > > > > > > read(2) takes a size_t so this is clearly an internal bug where it's an int and treating it as a negative value. > > > > Not exactly. The reason for cutting it off are many fold. Using int in > > the kernel is one argument. The requirement for locking the IO range for > > concurrent read/write operations from other threads is a bigger > > argument. > > > That still doesn't justify EINVAL as a return. Does read(2) need to > make atomicity guarantees? See my first sentence. I consider returning EINVAL for too large buffer size a bug. Yes, read/write operations should be atomic with regard to other processes on the system. Atomic meaning in this context that the read can be observed either completely or not at all. This still doesn't mean that read must execute the full size. Other cases for short read/writes are socket, pipes etc. Joerg