From owner-freebsd-net Thu May 9 13:39:38 2002 Delivered-To: freebsd-net@freebsd.org Received: from mailout.informatik.tu-muenchen.de (mailout.informatik.tu-muenchen.de [131.159.0.5]) by hub.freebsd.org (Postfix) with ESMTP id 0E06B37B406 for ; Thu, 9 May 2002 13:39:32 -0700 (PDT) Received: from mailrelay1.informatik.tu-muenchen.de (mailrelay1.informatik.tu-muenchen.de [131.159.254.5]) by mailout.informatik.tu-muenchen.de (Postfix) with ESMTP id EAE5D6135 for ; Thu, 9 May 2002 22:39:30 +0200 (MEST) Received: from atrbg11.informatik.tu-muenchen.de (atrbg11.informatik.tu-muenchen.de [131.159.24.91]) by mailrelay1.informatik.tu-muenchen.de (Postfix) with ESMTP id BB1707942 for ; Thu, 9 May 2002 22:39:30 +0200 (MEST) Received: by atrbg11.informatik.tu-muenchen.de (Postfix, from userid 20455) id 7671D139C8; Thu, 9 May 2002 22:39:30 +0200 (CEST) Date: Thu, 9 May 2002 22:39:30 +0200 From: Daniel Lang To: freebsd-net@freebsd.org Subject: Multiple NICs on the same subnet Message-ID: <20020509203930.GA32900@atrbg11.informatik.tu-muenchen.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.27i X-Geek: GCS/CC d-- s: a- C++$ UBS++++$ P+++$ L- E-(---) W+++(--) N++ o K w--- O? M? V? PS+(++) PE--(+) Y+ PGP+ t++ 5+++ X R+(-) tv+ b+ DI++ D++ G++ e+++ h---(-) r++>+++ y+ Sender: owner-freebsd-net@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Hi, I want to use multiple NICs on the same subnet. Thats my setup: FreeBSD 4.5-STABLE (updated 2-3 weeks ago, no 4.6-PRERELEASE yet), 3 NICs inside: 2 x 3com 905C-TX, 1 x D-Link DGE-500SX (Level 1 Gigabit). Now I want to assign a couple of IP addresses to those NICs. All IP addresses are on the same subnet, and they correspond to different kind of services available from this host, like FTP, CVSup, http, rsync, etc. Previously all services have been served by a single NIC, and I've used IP aliases to assign the service bound addresses to the NIC. Now I want to assign a certain set of services to different NICs (which all have separate uplinks at their switches and thus can achieve more bandwidth), like bind the FTP service to the lge interface, and maybe rsync and CVSup to one of the 3coms and the rest to the remaining 3com. This does not seem to be possible. If I want to assign an address to one of those NICs, I get an error from an ioctl like "file exists". Reading the archives, etc, it seems like I need to assign this address as an alias. This sort of worked. I could assign an address to one of the other NICs by choosing a netmask of /32 and stating alias on using ifconfig. But, our switch did not register any packets from one of the other NICs other than the default interface (one of the 3coms). It seems, like the address may be assigned to one of the NICs, but even if I use an application which explicitly binds to a certain source IP, the packet does not get sent by the interface, which has this IP address assigned, but by the default interface. This would not be that much of a problem so far. The problem really showed up, when it seemed like the Gigabit interface did not seem to work as expected. A couple of possible problems may be the cause, symptoms beeing "lge0: watchdog timeout" messages (which may be due to hardware/cabling problems), "sendto: no buffer space availble" messages (no idea where this comes from, any hints appreaciated, kern.ipc.nmbclusters and kern.maxusers etc, are bumped enough and did not max out (according to netstat -m)). So I thought, I run the production services on the reliable xl0 and xl1 interfaces and assign a test IP address to the lge0 interface, and if services from that interface run reliable, too, I switch the others over, benefitting from the increased bandwith. Well, thats how I would like it to work. But it doesn't. Is there any chance in FreeBSD to do it, and if, how? I've heard some rumours, it is not possible to use multiple NICs on the same subnet, since the IP stack would not know which interface to use to transmit packets, since it could not use its routing table (as the network is the same). But my argument would be, of course it should use the interface, which was configured with the address, the service bound its source port to. If any service did bind to IN_ADDR_ANY (0.0.0.0), the ip stack may choose the default interface or just any interface to transmit the packet from. I've also heard Bill Paul did some work on fat ethernet channels, which may be an implementation of Ciscos ethernet channel trunking/bundling, which also requires support from the switching counterpart. Although this is very interesting it does not help me at all, since I do not intend to bundle my NICs to a single fat channel, but to assign certain services to it, and to do some testing, but all on the same subnet. Any help greatly appreciated. Best regards, Daniel -- IRCnet: Mr-Spock - "I hear that, if you play the WindowsXP CD backwards, you get a Satanic message!" - "That's nothing. If you play it forward, it installs WindowsXP!" Daniel Lang * dl@leo.org * +49 89 289 25735 * http://www.leo.org/~dl/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message