From owner-freebsd-hackers Tue Apr 23 10:37:51 2002 Delivered-To: freebsd-hackers@freebsd.org Received: from gull.prod.itd.earthlink.net (gull.mail.pas.earthlink.net [207.217.120.84]) by hub.freebsd.org (Postfix) with ESMTP id 0C99537B416 for ; Tue, 23 Apr 2002 10:37:46 -0700 (PDT) Received: from pool0547.cvx22-bradley.dialup.earthlink.net ([209.179.200.37] helo=mindspring.com) by gull.prod.itd.earthlink.net with esmtp (Exim 3.33 #2) id 1704EV-0007Rb-00; Tue, 23 Apr 2002 10:37:39 -0700 Message-ID: <3CC59BC7.1AA75183@mindspring.com> Date: Tue, 23 Apr 2002 10:37:11 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Attila Nagy Cc: hackers@freebsd.org Subject: Re: sendfile() in tftpd? References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Attila Nagy wrote: > > Only if all file transfers were binary, or all ASCII files were stored > > on the host with line termination, instead of . That's the > > same reason sendfile() is stupid for POP3, IMAP4, and SMTP servers... > > Hmm. Both FTP and TFTP supports ASCII and binary transfers, am I right? TFTP... sendfile() doesn't work with UDP, anyway. But anyway... by definition, sendfile can't do the requisite end of line terminator conversion ( fro Macintosh, for UNIX, for Windows/MS-DOS) for FTP/TFTP, and it can't do the end of line conversion for message stores or mail transfer (SMTP, POP3, and IMAP4 all specify that lines *shall* be terminated by ). > In libexec/ftpd this case is handled this way: > case TYPE_A: > while ((c = getc(instr)) != EOF) { > [...] > case TYPE_L: > [...] > while (err != -1 && filesize > 0) { > err = sendfile(filefd, netfd, offset, 0, > (struct sf_hdtr *) NULL, &cnt, 0); > [...] > > As far as I can determine. Yep. Wouldn't it be nice if it always worked, instead of only working for binary? > In libexec/tftpd there is also a similar possibility to handle each case, > because there is "netascii" and "octet". Yeah, but the UDP lets it out, anyway. [ ... TFTP booting ... ] > And that's what isn't too good. One client can achieve about 15 Mbps but > if more than 10 (usually 20-30) clients want to fetch the images the TFTP > server first slows down (it eats all the CPU of the server, which is a > fast AthlonXP 1600+) then times out. The answer is probably "boot a small image, and use NFS for the real data, so you don't have to use UDP for the bulk of the data you have to transfer". > I think this could be improved if TFTP could use sendfile(). > (I have a busy FTP server and I know how much sendfile() can speed up > things) FTP is TCP, not UDP. TFTP "magically" implements session state (retransmit/retry) on top of UDP. In other words, it basically implements the stream delivery you would get with TCP, in user space. > The main question here seems to be: could sendfile() be used with UDP, or > it is just for TCP? It could probably be used. The retransmits, though, really need to be built into the protocol, so it's pretty useless for UDP, if ever you drop a signle packet, since you won't be able to recover. You would have to implement the code for it. Probably, the best idea, if you insist on it, is to implement the TFTP retransmit/acknowledge for the reliable data delivery in the kernel... as a stream protocol layer on top of UDP. And then implement sendfile on top of that. > BTW, the bpbatch stuff uses binary transfer (according to tcpdump > output)... All that means is that you won't have to do data conversion in that particular case. UDP datagrams don't really have a window acknowledgement (UDP datagram delivery is, by definition, unreliable), so you can't really self-clock the buffer size on the receiver to ensure that you don't have to do retransmits. You *might* be able to get around this with the "TFTP as a protocol layer" hack, but the window size is one packet, so you're not really going to see a significant speed improvement, after all the hacking is said and done. Maybe 12%. The main win of sendfile() at all is in not eating the window fill latency by going back to user space, when using sendfile() with TCP; actually, the overhead savings for using sendfile() can be had without using sendfile(), for the most port, since the window is generally large enough and the consumption slow enough that you end up disk-bound on the sender, or ack-paced by the receiver's inability to drain the buffer quickly, so you only ever see two round trip latencies over the whole data stream. For UDP, you're going to see the round trip latency for each packet for the individual ACK's, so you're pretty much screwed from the start, if you use a non-wondowed protocol like TFTP over UDP, at all. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message