From owner-freebsd-net@freebsd.org Wed Jan 17 21:56:26 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5DC12EBA4AC for ; Wed, 17 Jan 2018 21:56:26 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-qt0-x22b.google.com (mail-qt0-x22b.google.com [IPv6:2607:f8b0:400d:c0d::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1B07C780 for ; Wed, 17 Jan 2018 21:56:26 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-qt0-x22b.google.com with SMTP id 33so25440740qtv.1 for ; Wed, 17 Jan 2018 13:56:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=AXndqsrzv6aR7w3VkwveUQMJzrky2rMqkYZHL1MMlWM=; b=IGsgshXnFacDfva4N4pG4lsdOWQFouDjlSd+5ThRW0T1Tw7siVInZvSBvK+vTPi//K 7SC31k5aKKVutaDjpgxNjHzctedHiAMjbrOOU4cqa681MxU477B5w68UixLep0f56mGV 4ZX0BG5TuEpijx+PpTSWQZcdvjaQ+ZsKWM0P/QI1336y5HnaFiIj0AVG6ZaTeBVkWw/N 5b4SNws0Dn4gammjDehTpDv/iM4JB9spLaNf5u3J4e8ncloxWqCKDn9n80tfH5Y+Im5Y wphIutNq0TDi9RVVWSPNuVP+qjBmrfUWw0EsJoFSR4x7oT2zrrjdkd8xPubbZyljU1RH VVIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=AXndqsrzv6aR7w3VkwveUQMJzrky2rMqkYZHL1MMlWM=; b=Fpz+yoQrEMiqWdiUyesNLAp0T6x+9EHEc2Su04BfSFoXT2JtPE81FiADRzC3x5t3WS UXtXMrSNU1ZP3rJ+mLszGWgYudkyxz8FE0g0uB0E7raiBweSYF9NdNjByqx+/cZao9PR umxnvKYDzxu//pwmcIWOqIe7ratgHdo7eukT5kwPZJvLXhfS34ohg+G6WprEoZy/dTD3 QCRxmaLfHlDrwEsRu/0RO+1ZgugOz9pluekz3xdNuymMYAK2nxSFXJYI1F4hDYfCofgV CpDSuDUvQkSt4b3YQIszudNJmI4nuN856tz6nZpMU4W+TKjdJoGx+hka7D4y39NO01lJ Cb8A== X-Gm-Message-State: AKwxytcuGZJVvxHQ+d3K3EIB/MOZ7QjHKZA5CDG9rzyLoMPZJvgDOK1v sSWNbxR5blJikKf8eHGdmaiN3nm7MNuspsfI0EWduQ== X-Google-Smtp-Source: ACJfBouKUqEt0yjCMOBZ6honmrImOwj3z1O3sK3XsOiu+sWFDIYUswAhacljFmTGjiu0sLW9Qa2LpWccKITeM2RJ1KE= X-Received: by 10.200.23.136 with SMTP id o8mr40386581qtj.255.1516226184890; Wed, 17 Jan 2018 13:56:24 -0800 (PST) MIME-Version: 1.0 Received: by 10.237.39.136 with HTTP; Wed, 17 Jan 2018 13:56:24 -0800 (PST) From: Ryan Stone Date: Wed, 17 Jan 2018 16:56:24 -0500 Message-ID: Subject: Allowing a local subnet route to change to a different ifnet To: freebsd-net Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2018 21:56:26 -0000 I'm going to prefix this question by noting that I realize that the configuration that I am about to describe is quite nonsensical. Unfortunately, it seems that under older versions of FreeBSD (possibly FreeBSD 7-vintage), the configuration mostly "worked", and now at $WORK I am responsible for making the configuration continue to work in the future. The customer in this case is completely unwilling to modify their configuration. I have a customer that has configured two different IPs on the same subnet on two different interfaces. The behaviour that they want is that if the link on one of the two interfaces goes down, the route to that subnet will migrate to the other IP on the other interface as a quasi-failover behaviour. Under FreeBSD 7, we had a daemon that accomplished this by detecting the link loss and then using "route change" to move the route to the up interface. If the subnet in question was 192.168.1.0/24, for example, we could run "route change 192.1.68.1.0/24 -ifp em1" to migrate the route. Running on -head I run into two issues. The first comes out of r264986, which changes the behaviour of RTM_CHANGE. The code path changed significantly, but the part that impacts me is that now any RTM_CHANGE command with the gateway set NULL gets EINVAL immediately where previously it was allowed. I've hacked around this problem locally for testing purposes but I really don't understand the code well enough at this point to see what a real fix would look like. The second issue is quite complex. The root cause is that the routing code sets IFA_ROUTE on any ifaddr that has a local subnet route associated with it. If I don't migrate this flag to the new ifaddr, very bizarre behaviour follows. I've done a prototype that detects when this migration is needed in rtrequest1_fib_change() and it works, but IFA_ROUTE seems to otherwise be handled in individual L3 protocols like in and in6, so I'm worried that this is a layering violation. My patch looks like this: https://people.freebsd.org/~rstone/patches/route-change-subnet.diff My first, and most important question, is whether a patch that would allow a subnet route to be migrated to a different interface be something that would be acceptable in FreeBSD? If so, I need guidance on what a proper fix for both issues would look like so that I can implement fixes that I can upstream. Thanks, Ryan