From owner-freebsd-net@FreeBSD.ORG Sun Sep 12 14:58:43 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AAD2E106564A for ; Sun, 12 Sep 2010 14:58:43 +0000 (UTC) (envelope-from manishv@lineratesystems.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 4BF798FC13 for ; Sun, 12 Sep 2010 14:58:43 +0000 (UTC) Received: by fxm4 with SMTP id 4so3145083fxm.13 for ; Sun, 12 Sep 2010 07:58:42 -0700 (PDT) MIME-Version: 1.0 Received: by 10.223.123.68 with SMTP id o4mr319310far.76.1284302022864; Sun, 12 Sep 2010 07:33:42 -0700 (PDT) Received: by 10.223.112.16 with HTTP; Sun, 12 Sep 2010 07:33:42 -0700 (PDT) Date: Sun, 12 Sep 2010 08:33:42 -0600 Message-ID: From: Manish Vachharajani To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: No ifp->if_ioctl for SIOCAIFADDR and SIOCDIFADDR causes CARP issues on 7.3? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Sep 2010 14:58:43 -0000 I was poking around some of the in_carp.c code and noticed some odd behavior due to the fact that ifp->if_ioctl is not always called on address configuration. It seems that ifp->if_ioctl is not called for SIOCAIFADDR and SIOCDIFADDR, only SIOCSIFADDR under FreeBSD 7.3 release. I confirmed this with a few debug printfs. This results in a few bugs: 1) Improper advertisement even when no IPs are configured Since carp_del_addr is what decrements sc->naddrs, carp will continue to advertise (with the correct HMAC, if there is only one IP, since the carp_hmac_prepare is not called on address deletion) even when there are no IP addresses on the interface. As a result, you can end up with a master system that will not actually serve for the given IP when packets arrive. To cause this, just do ifconfig carp0 create ifconfig carp0 vhid 10 advskew 0 ifconfig carp0 ifconfig -a # confirm that the carp0 interface is MASTER ifconfig carp0 delete tcpdump -i proto carp #notice the advertisements continue ifconfig carp 0 destroy #now the advertisements stops and the physical interface leaves promiscuous mode, confirm via dmesg and above tcpdump Note further that whatever physical interface is associated with the carp device will stay in promiscuous mode until the carp interface is destroyed for the same reason. 2) Improper operation with multiple IPs per CARP interface if IPs are added in the wrong order Carp will work with one virtual ip per carp interface but causes it to function improperly with multiple IPs. For example, it appears that carp will compute the wrong HMAC if IPs are added in different orders on different machines since the hmac is only updated for the first IP address in the group (set with SIOCSADDR by ifconfig). However, carp_hmac_prepare will do the right thing, if only it were called on SIOCAIFADDR and DIFADDR. 3) Not interoperable with multiple IPs (compared to other CARP implementations) Any CARP implementation (such as OpenBSD's which appears to do the right thing) will not interoperate with FreeBSD's (though I haven't confirmed this by booting OpenBSD and trying it) if multiple IPs are configured because FreeBSD only computes the HMAC based on the first IP address that was added, even though carp_hmac_prepare has code to correctly recompute the hmac when new IPs are added in an address independent fashion. It is just that the function is never called since the update would normally be through ifp->if_ioctl. According to the man page for ifnet(9), this is the correct behavior, if_ioctl is only called for SIOCSIFADDR, not AIFADDR and DIFADDR. How should a driver go about listening for these events if not through ifp->if_ioctl? Manish