Date: Thu, 15 Mar 2001 14:17:21 +0100 (CET) From: Jean-Luc.Richier@imag.fr To: Jean-Luc.Richier@imag.fr Subject: bin/25826: nfsd -h option dosen't work in TCP Message-ID: <200103151317.f2FDHLo00698@tuna.imag.fr>
next in thread | raw e-mail | index | archive | help
>Number: 25826 >Category: bin >Synopsis: nfsd -t -h adr1 -h adr2 doesn't work >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Thu Mar 15 05:30:01 PST 2001 >Closed-Date: >Last-Modified: >Originator: Jean-Luc Richier >Release: FreeBSD 4.2-RELEASE i386 >Organization: LSR-IMAG >Environment: Any FreeBSD >=4.2 (I checked stable and current on March 14, 2001) >Description: On a nfs server with >=2 IP addresses, nfsd server being configured with -h options, as recommended in man nfsd, mount_nfs accesses over TCP will fail. The problem is the same when using -a option. >How-To-Repeat: Consider a nfs server with two IP addresses, xx and yy. According to the nsfd man, set the following nfsd options in rc.conf: nfs_server_enable="YES" nfs_server_flags="-u -t -n 4 -h xx -h yy and export some catalog (e.g. /usr/local) On an other machine, try to mount xx:/usr/local over TCP. The mount command will fail: % mount -o -T xx:/usr/local /mnt nfs server xx:/usr/local: not responding Also nfsd on xx is now burning CPU: # ps -ax | grep nfsd 545 ?? Rs 0:23.78 nfsd: master (nfsd) 546 ?? I 0:00.00 nfsd: server (nfsd) 547 ?? I 0:00.00 nfsd: server (nfsd) 548 ?? I 0:00.00 nfsd: server (nfsd) 549 ?? I 0:00.00 nfsd: server (nfsd) # sleep 4 # ps -ax | grep nfsd 545 ?? Rs 0:27.81 nfsd: master (nfsd) 546 ?? I 0:00.00 nfsd: server (nfsd) 547 ?? I 0:00.00 nfsd: server (nfsd) 548 ?? I 0:00.00 nfsd: server (nfsd) 549 ?? I 0:00.00 nfsd: server (nfsd) Notes: Mounting using UDP works. TCP mount of yy:/usr/local (the last address in the -h list) works but only if no TCP mount has failed: % mount -o -Ti yy:/usr/local /mnt .... mount OK % umount /mnt % mount -o -Ti xx:/usr/local /mnt nfs server xx:/usr/local: not responding ^C % mount -o -Ti yy:/usr/local /mnt ... no response >Fix: >Release-Note: >Audit-Trail: >Unformatted: >Analysis: The problem is in the nfsd.c code: with -h option, multiple tcp sockets are created, one for each address 470 /* Now set up the master server socket waiting for tcp connections. */ 471 on = 1; 472 FD_ZERO(&sockbits); 473 connect_type_cnt = 0; 474 for (i = 0; tcpflag && i < bindhostc; i++) { 475 if ((tcpsock = socket(AF_INET, SOCK_STREAM, 0)) < 0) { ... 482 setbindhost(&inetaddr, bindhost[i]); 483 if (bind(tcpsock, 484 (struct sockaddr *)&inetaddr, sizeof (inetaddr)) < 0) { ... 488 if (listen(tcpsock, 5) < 0) { 489 syslog(LOG_ERR, "listen failed"); 490 exit(1); 491 } ..... 500 } However in the main loop which accept the tcp connections, the code tests only one socket (the last one kept in tcpsock): 581 /* 582 * Loop forever accepting connections and passing the sockets 583 * into the kernel for the mounts. 584 */ 585 for (;;) { 586 ready = sockbits; 587 if (connect_type_cnt > 1) { 588 if (select(maxsock + 1, .... 593 } 594 if (tcpflag && FD_ISSET(tcpsock, &ready)) { 595 len = sizeof(inetpeer); 596 if ((msgsock = accept(tcpsock, 597 (struct sockaddr *)&inetpeer, &len)) < 0) { 598 syslog(LOG_ERR, "accept failed: %m"); 599 exit(1); There is no loop for the multiple possible values of tcpsock Fix: There is different possible fixes 1/ the simplest fix does not consider the tp4 tpip code (which is not used, being between #ifdef notyet directives.) Simply loop on all possible values of tpcsock: *** nfsd.c.DIST Sun Sep 17 00:52:23 2000 --- nfsd.c Thu Mar 15 13:58:58 2001 *************** *** 591,597 **** exit(1); } } ! if (tcpflag && FD_ISSET(tcpsock, &ready)) { len = sizeof(inetpeer); if ((msgsock = accept(tcpsock, (struct sockaddr *)&inetpeer, &len)) < 0) { --- 591,602 ---- exit(1); } } ! for (tcpsock = 0; tcpsock <= maxsock; tcpsock++) { ! if (tcpflag && FD_ISSET(tcpsock, &ready)) { ! #ifdef notyet ! XXX - should test that tcpsock is indeed a tcp socket ! and not the tp4 socket or a tpip socket ! #endif /* notyet */ len = sizeof(inetpeer); if ((msgsock = accept(tcpsock, (struct sockaddr *)&inetpeer, &len)) < 0) { *************** *** 608,613 **** --- 613,619 ---- nfsdargs.namelen = sizeof(inetpeer); nfssvc(NFSSVC_ADDSOCK, &nfsdargs); (void)close(msgsock); + } } #ifdef notyet if (tp4flag && FD_ISSET(tp4sock, &ready)) { 2/ If we want to correct the tpip/tp4 code, one has to add 2 fd_set: fd_set tcpbits, tpipbits; to set the bits in the fd_set in parallel to the bits in sockbits, and to use them to test is a selected socket is a tcp socket or a tpip socket (there is only one tp4socket, so the old code wille work); 3/ And other solution is to have only one tcp socket: The -h optionhas 2 uses: - One is to avoid and udp specific problem - cf the man: If nfsd is to be run on a host with multiple interfaces or interface aliases, use of the -h option is recommended. If you do not use the op- tion NFS may not respond to UDP packets from the same IP address they were sent to. - The other is to control nfs access - cf the man: Use of this option is also recommended when securing NFS exports on a firewalling machine such that the NFS sockets can only be accessed by the inside interface. Ipfw would then be used to block nfs- related packets that come in on the outside interface. But filtering may be succifient. Therefore multiple tcp sockets are much less useful than udp ones. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-bugs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200103151317.f2FDHLo00698>