Date: Tue, 23 Apr 2002 10:37:11 -0700 From: Terry Lambert <tlambert2@mindspring.com> To: Attila Nagy <bra@fsn.hu> Cc: hackers@freebsd.org Subject: Re: sendfile() in tftpd? Message-ID: <3CC59BC7.1AA75183@mindspring.com> References: <Pine.LNX.4.44.0204231427560.24266-100000@scribble.fsn.hu>
next in thread | previous in thread | raw e-mail | index | archive | help
Attila Nagy wrote:
> > Only if all file transfers were binary, or all ASCII files were stored
> > on the host with <CR><LF> line termination, instead of <LF>. That's the
> > same reason sendfile() is stupid for POP3, IMAP4, and SMTP servers...
>
> Hmm. Both FTP and TFTP supports ASCII and binary transfers, am I right?
TFTP... sendfile() doesn't work with UDP, anyway.
But anyway... by definition, sendfile can't do the requisite end of
line terminator conversion (<CR> fro Macintosh, <LF> for UNIX, <CR><LF>
for Windows/MS-DOS) for FTP/TFTP, and it can't do the end of line
conversion for message stores or mail transfer (SMTP, POP3, and IMAP4
all specify that lines *shall* be terminated by <CR><LF>).
> In libexec/ftpd this case is handled this way:
> case TYPE_A:
> while ((c = getc(instr)) != EOF) {
> [...]
> case TYPE_L:
> [...]
> while (err != -1 && filesize > 0) {
> err = sendfile(filefd, netfd, offset, 0,
> (struct sf_hdtr *) NULL, &cnt, 0);
> [...]
>
> As far as I can determine.
Yep. Wouldn't it be nice if it always worked, instead of only
working for binary?
> In libexec/tftpd there is also a similar possibility to handle each case,
> because there is "netascii" and "octet".
Yeah, but the UDP lets it out, anyway.
[ ... TFTP booting ... ]
> And that's what isn't too good. One client can achieve about 15 Mbps but
> if more than 10 (usually 20-30) clients want to fetch the images the TFTP
> server first slows down (it eats all the CPU of the server, which is a
> fast AthlonXP 1600+) then times out.
The answer is probably "boot a small image, and use NFS for the
real data, so you don't have to use UDP for the bulk of the data
you have to transfer".
> I think this could be improved if TFTP could use sendfile().
> (I have a busy FTP server and I know how much sendfile() can speed up
> things)
FTP is TCP, not UDP. TFTP "magically" implements session state
(retransmit/retry) on top of UDP. In other words, it basically
implements the stream delivery you would get with TCP, in user space.
> The main question here seems to be: could sendfile() be used with UDP, or
> it is just for TCP?
It could probably be used. The retransmits, though, really need
to be built into the protocol, so it's pretty useless for UDP, if
ever you drop a signle packet, since you won't be able to recover.
You would have to implement the code for it.
Probably, the best idea, if you insist on it, is to implement the
TFTP retransmit/acknowledge for the reliable data delivery in the
kernel... as a stream protocol layer on top of UDP. And then
implement sendfile on top of that.
> BTW, the bpbatch stuff uses binary transfer (according to tcpdump
> output)...
All that means is that you won't have to do data conversion in
that particular case.
UDP datagrams don't really have a window acknowledgement (UDP
datagram delivery is, by definition, unreliable), so you can't
really self-clock the buffer size on the receiver to ensure that
you don't have to do retransmits. You *might* be able to get
around this with the "TFTP as a protocol layer" hack, but the
window size is one packet, so you're not really going to see a
significant speed improvement, after all the hacking is said and
done. Maybe 12%.
The main win of sendfile() at all is in not eating the window fill
latency by going back to user space, when using sendfile() with
TCP; actually, the overhead savings for using sendfile() can be
had without using sendfile(), for the most port, since the window
is generally large enough and the consumption slow enough that you
end up disk-bound on the sender, or ack-paced by the receiver's
inability to drain the buffer quickly, so you only ever see two
round trip latencies over the whole data stream.
For UDP, you're going to see the round trip latency for each packet
for the individual ACK's, so you're pretty much screwed from the
start, if you use a non-wondowed protocol like TFTP over UDP, at all.
-- Terry
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3CC59BC7.1AA75183>