From owner-freebsd-hackers Tue Apr 27 2:35:11 1999 Delivered-To: freebsd-hackers@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [209.157.86.2]) by hub.freebsd.org (Postfix) with ESMTP id BE1CB14CD4 for ; Tue, 27 Apr 1999 02:35:07 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id CAA00567; Tue, 27 Apr 1999 02:35:05 -0700 (PDT) (envelope-from dillon) Date: Tue, 27 Apr 1999 02:35:05 -0700 (PDT) From: Matthew Dillon Message-Id: <199904270935.CAA00567@apollo.backplane.com> To: David Greenman , Julian Elischer , luoqi@watermarkgroup.com, alc@cs.rice.edu, hackers@freebsd.org Subject: Found TCP NFS bug in packet trace. Looks like nfs_realign() is broken Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I direct everyone's attention to the '30fb 64XX' sequence in the below packet trace. This demonstrates why TCP NFS links get stuck. The TCP packet is encompassing *two* requests rather then one. This is correct. However, the server is screwing up on it. Note that in the NFS request, the NFS xid is '30fb 64dc' for the first request and '30fb 64dd' for the second. Note that in the NFS reply, *BOTH* xid's in the reply are 30fb 64dd. 30fb 64dc got destroyed! In fact, it looks like the server is sending two copies of the 30fb 64dd response instead of a 30fb 64dc and 30fb 64dd response. I tracked it down to nfs_realign(). The routine tries to realign data contained in an mbuf. It completely destroys one of the commands in the mbuf while attempting to realign the other. The destruction occurs due to the shared-data nature of mbufs when ( as far as I can tell ) the mbuf is flagged M_EXT. If I turn nfs_realign() into a NOP, TCP mounts appear to work again ( at least better then they did before. There is still occassional weirdness ). Doh! I'll have a patch to fix this properly later on tuesday. This brings up another problem, though - unaligned mbufs appear to be a regular occurance w/NFS but the data *appears* to be aligned in the tcpdump. Is the MAC header or TCP options causing the packet data to be unaligned? I'm guessing that NFS is doing a lot of buffer realigning and its probably having a negative effect on performance. I can rewrite the alignment code to allocate fresh mbufs, but I would kinda like to know why the below tcp packets appear to be unaligned from the point of view of the NFS code. there might be a quick fix we can make somewhere to increase performance. -Matt Matthew Dillon 23:48:09.209937 209.157.86.12.1021 > 209.157.86.2.2049: P 100656:100920(264) ack 60585 win 17200 (DF) 4500 0130 1436 4000 4006 d648 d19d 560c d19d 5602 03fd 0801 186c d0c5 7371 bbce 5018 4330 3ab3 0000 8000 0080 30fb 64dc <<<<<<<<< 0000 0000 0000 0002 0001 86a3 0000 0003 0000 0003 0000 0001 0000 0030 0000 0000 0000 0000 0000 0000 0000 0000 0000 0007 0000 0000 0000 0002 0000 0003 0000 0004 0000 0005 0000 0014 0000 001f 0000 0000 0000 0000 0000 001c 0700 0000 4216 9c77 0c00 0000 8e45 0300 6922 8679 0000 0000 0000 0000 0000 0004 6c69 623a 8000 0080 >>>>>>>> 30fb 64dd 0000 0000 0000 0002 0001 86a3 0000 0003 0000 0003 0000 0001 0000 0030 0000 0000 0000 0000 0000 0000 0000 0000 0000 0007 0000 0000 0000 0002 0000 0003 0000 0004 0000 0005 0000 0014 0000 001f 0000 0000 0000 0000 0000 001c 0700 0000 4216 9c77 0c00 0000 8e45 0300 6922 8679 0000 0000 0000 0000 0000 0003 6c69 6200 23:48:09.214561 209.157.86.2.2049 > 209.157.86.12.1021: P 60749:60989(240) ack 101116 win 33176 (DF) 4500 0118 7cdc 4000 4006 6dba d19d 5602 d19d 560c 0801 03fd 7371 bc72 186c d291 5018 8198 7102 0000 8000 0074 30fb 64dd <<<<<<<< 0000 0001 0000 0000 0000 0000 0000 0000 0000 0000 0000 0002 0000 0001 0000 0002 0000 01ed 0000 0003 0000 0000 0000 0000 0000 0000 0000 0200 0000 0000 0000 0400 0000 00c6 000d 0016 0000 0000 0000 0007 0000 0000 0003 458e 3725 5d9d 0000 0000 3725 5d9d 0000 0000 3725 5d9d 0000 0000 >>>>>>>> 8000 0074 30fb 64dd 0000 0001 0000 0000 0000 0000 0000 0000 0000 0000 0000 0002 0000 0001 0000 0002 0000 01ed 0000 0003 0000 0000 0000 0000 0000 0000 0000 0200 0000 0000 0000 0400 0000 00c6 000d 0016 0000 0000 0000 0007 0000 0000 0003 458e 3725 5d9d 0000 0000 3725 5d9d 0000 0000 3725 5d9d 0000 0000 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message