From owner-freebsd-hackers Fri Nov 30 13:30:44 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 28CD637B417 for ; Fri, 30 Nov 2001 13:30:31 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id fAULUU324648; Fri, 30 Nov 2001 13:30:30 -0800 (PST) (envelope-from dillon) Date: Fri, 30 Nov 2001 13:30:30 -0800 (PST) From: Matthew Dillon Message-Id: <200111302130.fAULUU324648@apollo.backplane.com> To: Alexander Haderer , freebsd-hackers@FreeBSD.ORG Subject: Found the problem, w/patch (was Re: FreeBSD performing worse than Linux?) References: <20011128153817.T61580@monorchid.lemis.com> <15364.38174.938500.946169@caddis.yogotech.com> <20011128104629.A43642@walton.maths.tcd.ie> <5.1.0.14.1.20011130181236.00a80160@postamt1.charite.de> <200111302047.fAUKlT811090@apollo.backplane.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I believe I have found the problem. The transmit side has a maximum burst count imposed by newreno. As far as I can tell, if this maxburst is hit (it defaults to 4 packets), the transmitter just stops - presumably until it receives an ack. Now, theoretically this should work just fine... send four packets, receive the first ack and send the next four packets... it should allow us to fill the window geometrically. I believe the idea is to give transmit packets a chance to include acks for received data in a reasonable period of time... I'm not sure, it's J Lemon's commit (from the original newreno commits) so maybe he can work it out. However, if the receiver has delayed-acks turned on only one ack is returned for all four packets. The next four are then sent and one ack is returned. I believe this the cause of the problem. It effectively destroys the TCP window, forcing it to around 1.5Kx4 = 6K. This also explains why performance is so weird... if more then one delayed ack happens to occur per burst you get 'bumps' in the performance. Without the patch, two things will solve or partially solve the problem: * Turn off delayed acks on the receiver (performance 80K->6.8MB/sec) OR * Turn off newreno on the transmitter. (performance 80K->7.9MB/sec) The patch below kills the burst limit on the transmit side and appears to solve the problem permanently. I'm sure I'm breaking something in the newreno RFC, but I am going to commit it to both branches now because our current situation is horrible. -Matt Index: tcp_output.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/tcp_output.c,v retrieving revision 1.39.2.10 diff -u -r1.39.2.10 tcp_output.c --- tcp_output.c 2001/07/07 04:30:38 1.39.2.10 +++ tcp_output.c 2001/11/30 21:18:10 @@ -912,7 +912,14 @@ tp->t_flags &= ~TF_ACKNOW; if (tcp_delack_enabled) callout_stop(tp->tt_delack); +#if 0 + /* + * This completely breaks TCP if newreno is turned on + */ if (sendalot && (!tcp_do_newreno || --maxburst)) + goto again; +#endif + if (sendalot) goto again; return (0); } To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message