From owner-freebsd-arch@FreeBSD.ORG Tue May 27 03:49:39 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 23B8437B401 for ; Tue, 27 May 2003 03:49:39 -0700 (PDT) Received: from park.rambler.ru (park.rambler.ru [81.19.64.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id D191443FAF for ; Tue, 27 May 2003 03:49:37 -0700 (PDT) (envelope-from is@rambler-co.ru) Received: from is.park.rambler.ru (is.park.rambler.ru [81.19.64.102]) by park.rambler.ru (8.12.6/8.12.6) with ESMTP id h4RAnZmF015243; Tue, 27 May 2003 14:49:35 +0400 (MSD) Date: Tue, 27 May 2003 14:49:35 +0400 (MSD) From: Igor Sysoev X-Sender: is@is To: Terry Lambert In-Reply-To: <3ED33487.59C03DA9@mindspring.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: sendfile(2) SF_NOPUSH flag proposal X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 May 2003 10:49:39 -0000 On Tue, 27 May 2003, Terry Lambert wrote: > Igor Sysoev wrote: > > > I don't see this as being terrifically useful; small files > > > should probably just be mapped and written; the copy expense > > > is still there for the headers and trailers, no matter what, > > > and the file size itself is very small overhead, relatively > > > speaking, for files small enough for this to be an issue. > > > > FreeBSD 4.x has no zero_copy(9) so mmap()ed files would be copied. > > sendfile() allows to avoid this copy. > > It's actually "one copy", from an external mbuf reference > to a VM buffer to the card, vs. two copies, from the mapped > file in the process address space to mbufs, to the card. I know what "zero copy" means. > My point, though was that fr files small enough to fit in a > packet, you are probably going to spend more on the headers > and trailers copyin and the mbuf list assembly and the system > call overhead, than you'll end up spending one the extra data > copy for the very small file contents. > > I would be really surprised if you were able to demonstrate a > measuarble performance difference which was above the noise. I hope I will demonstrate at least CPU usage in near future. > > By the way what do you call by small files ? 4K, 30K or 100K ? > > You were talking about the file and the header living in the > same packet. I mean that if you have 230 bytes header then sendfile() will send it in separate packet nevertheless the size of header and of the file. Something like this - 230, 1460, 1460, ... > > > I also think your headers and trailers are very small, if > > > they are fitting with the file contents in a single packet. > > > I think this is atypical. > > > > If I ask Google: > > ---- > > HEAD /images/hp0.gif HTTP/1.0 > > Host: www.google.com > > > > ---- > > it will return me 230 bytes: > > The "HEAD" is atypical, compared to the "GET"; the full Google > front page is larger than that, and consists of multiple files; > assuming you support HTTP/1.1 and pipelining, it's going to be > a back-to-back transfer involving multiple sendfile() calls. I use HEAD to show you the size of the HTTP header. The HEAD is atypical but such small HTTP header is typical. > > > BTW: if you go ahead with this, you should verify that it > > > also works for the trailers, etc., and you should probably > > > skip it if you headers > transmit queue depth, or file size > > > > transmit queue depth, or trailers > transmit queue depth. > > > > Currently sendfile() can send the file in not full packets even > > file is bigger then the transmit queue depth. It can send > > it in 2 x 1460 + 1176 or 5 x 1460 + 892 packets. > > 3 packets vs. 6. And using HTTP/1.0, there's also the three > handshake packets, SYN/SYN-ACK/ACK, and the tear-down three > teardown packets, FIN/FIN-ACK/ACK (or 4), plus the ACK's for > the packets you sent (should be one ACK, since that's below > the TCP window size). Actually 6 vs. 6 for this 8K file. But I said about another thing. Let's see 48K file and 250 bytes header. sendfile() usually sends it as 4K or 8K hunks so there are 48/8 * 6 + 1 (header) = 37 packets. But (48K + 250) / 1460 = 33 * 1460 + 1270 i.e. 34 packets. It's 8% decrease of data packets. Add here the possible retransmitions. > Really: it's in the noise. Unless you are paying by packet > count, you probably shouldn't care. So do you consider that IP fragmentation is the good thing ? Igor Sysoev http://sysoev.ru/en/