Date: Tue, 24 Aug 1999 01:08:02 -0400 From: "Mark J. Taylor" <mtaylor@cybernet.com> To: wayne@crb-web.com Cc: hackers@freebsd.org Subject: Re: several messages Message-ID: <37C228B2.ECF2878E@cybernet.com> References: <Pine.LNX.3.95.990824011411.32533B-100000@crb.crb-web.com>
next in thread | previous in thread | raw e-mail | index | archive | help
As an (former) implementer of fast TCP/IP peer-peer communications, I'd have to agree with Dave, and say that it is definitely the TCP_NODELAY option. You'll find that disabling the TCP-ACK delay will greatly increase your performace. The reason that it is so "slow" is because the TCP/IP stack is trying to wait to send a TCP "ACK" piggy-backed with data that you MAY BE SENDING SOON. So it waits for 1/5 of a second for you to send SOMETHING, then shrugs its shoulders when you don't, and sends the TCP ACK without further delay. This is a "standard" that most TCP/IP stacks implement. You'll find the same "problem" under Windows. In fact, when I was doing the peer-peer communications, from a Unix to a Windows '95 machine (and NT), the TCP_NODELAY was not yet implemented in the WinSock library. The workaround was to send garbage back as fast as possible, so the ACK could piggy-back itself on SOMETHING. You may want to set the transmit and recieve low-water marks as well. Look at the man page for "setsockopt". -Mark Taylor NetMAX Developer mtaylor@cybernet.com http://www.netmax.com/ Wayne Cuddy wrote: > > Thank you for your reply. At what point should I set this socket option? I > am assuming right after the socket is allocated?? > > I will try this and post my results tomorrow night. > > For those wondering, I cannot just execute Sendmail directly, there are many > architectural reasons for this design... > > Thanks again, > > Wayne > > On Tue, 24 Aug 1999, Daniel O'Connor wrote: > > > Date: Tue, 24 Aug 1999 13:41:37 +0930 (CST) > > From: Daniel O'Connor <doconnor@gsoft.com.au> > > To: Wayne Cuddy <wayne@crb-web.com> > > Cc: FreeBSD Hackers List <freebsd-hackers@FreeBSD.ORG> > > Subject: RE: network performance vs. linux on small transfers > > > > > > On 24-Aug-99 Wayne Cuddy wrote: > > > I REALLY want to use FreeBSD over Linux on this one and need some major help > > > to get the performance out of FreeBSD. > > > > Tried setsockopt and TCP_NODELAY? > > > > >From netinet/tcp.h > > #define TCP_NODELAY 0x01 /* don't delay send to coalesce packets */ > > > > --- > > Daniel O'Connor software and network engineer > > for Genesis Software - http://www.gsoft.com.au > > "The nice thing about standards is that there > > are so many of them to choose from." > > -- Andrew Tanenbaum > > > > On Mon, 23 Aug 1999, David Greenman wrote: > > > Date: Mon, 23 Aug 1999 21:17:06 -0700 > > From: David Greenman <dg@root.com> > > To: wayne@crb-web.com > > Cc: FreeBSD Hackers List <freebsd-hackers@FreeBSD.ORG> > > Subject: Re: network performance vs. linux on small transfers > > > > >I am involved in a messaging system at work in which we need to send/receive > > >large amounts of small (one line messages) SMTP messages. We are currently using Sendmail 8.9.3 > > >on HPUX. > > > > > >Our application sends messages down a FIFO to a daemon process that is reading from > > >the FIFO. This process then connects to port 25 of the destination system and > > >delivers the mail via SMTP. Currently the destination system is the local > > >system so everything is done on one machine. > > > > > >Using HPUX we typically pass 5 messages a second. This system is a dual > > >180Mhz K class server so this is surprisingly low performance for this system. > > > > > >When testing on FreeBSD 3.1 we also got 5 messages a second. This system is a > > >500Mhz P3, this is also unacceptable performance. > > > > > >When we tested with Linux (kernel 2.2.5) we passed 15 messages a second > > >consistently using the exact same P3 described above. > > > > > >Since the HPUX and FreeBSD numbers are so close I am wondering there is some > > >performance tuning that I do not know about. Do you think the number might > > >change if multiple hosts were used? > > > > > >The daemon that reads from the FIFO makes only one connection to the local > > >Sendmail to deliver multiple messages in sequence. > > > > > > > > >I REALLY want to use FreeBSD over Linux on this one and need some major help > > >to get the performance out of FreeBSD. > > > > Are you setting the TCP_NODELAY socket option on the SMTP connection? If > > not, then please do that and let me know if it fixes the problem or not. > > > > -DG > > > > David Greenman > > Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org > > Creator of high-performance Internet servers - http://www.terasolutions.com > > Pave the road of life with opportunities. > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?37C228B2.ECF2878E>