Date: Sat, 13 Mar 1999 07:32:33 -0800 (PST) From: andrewb@demon.net To: freebsd-gnats-submit@freebsd.org Subject: kern/10570: Route delete with many routes on single interface causes panic Message-ID: <19990313153233.844A814D93@hub.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 10570
>Category: kern
>Synopsis: Route delete with many routes on single interface causes panic
>Confidential: no
>Severity: serious
>Priority: low
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Sat Mar 13 07:40:00 PST 1999
>Closed-Date:
>Last-Modified:
>Originator: Andrew Bangs
>Release: all
>Organization:
Demon Internet
>Environment:
>Description:
We've come across a problem that may affect people using *BSD boxes
as routers, particularly if they have a large route table where most
of the routes go via a single interface. We found this with OpenBSD,
but it looks to be a bit of common code going back to the Dark Ages.
The ifaddr structure (sys/net/if.h in NetBSD and OpenBSD; sys/net/if_var.h
in FreeBSD) looks something like this (this one from OpenBSD) :
/*
* The ifaddr structure contains information about one address
* of an interface. They are maintained by the different address families,
* are allocated and attached when an address is set, and are linked
* together so all addresses for an interface can be located.
*/
struct ifaddr {
struct sockaddr *ifa_addr; /* address of interface */
struct sockaddr *ifa_dstaddr; /* other end of p-to-p link */
#define ifa_broadaddr ifa_dstaddr /* broadcast address interface */
struct sockaddr *ifa_netmask; /* used to determine subnet */
struct ifnet *ifa_ifp; /* back-pointer to interface */
TAILQ_ENTRY(ifaddr) ifa_list; /* list of addresses for interface */
void (*ifa_rtrequest) /* check or clean routes (+ or -)'d */
__P((int, struct rtentry *, struct sockaddr *));
u_short ifa_flags; /* mostly rt_flags for cloning */
short ifa_refcnt; /* count of references */
int ifa_metric; /* cost of going out this interface */
};
See that 'short ifa_refcnt;' ?
How about making that a 'long ifa_refcnt;'?
Where people are using *BSD boxes as Internet routers, there are
around 56000 routes in a 'full route table', plus however many
might be used within an organisation's own network. We're doing
some particularly odd things, which leads to having around 75000
routes on some boxes. Even removing the 'odd things' that we're
doing with routes we would still be looking at around 60000 (and
this number is steadily growing).
The total number of routes isn't the problem, but if the number of
routes on a single interface goes above 2^16 and later falls below
2^16 then the reference counter hits zero and we free (IFAFREE in
if.h and ifafree() in route.c) the ifa. Subsequent route deletions
cause a panic was we flail around looking for a reference counter
to decrement in a structure that we've just freed (at least, that's
the behaviour in OpenBSD, and the code elsewhere looks the same).
I'm sending this to the Free, Net and OpenBSD communities in the
hope that the problem can be addressed before it bites anyone else.
>How-To-Repeat:
Add 2^16 routes all with destinations via the same interface.
Add another couple of routes with destination via the same interface.
Start deleting routes.
Watch it panic.
(We were using GateD rather than adding these manually.)
>Fix:
As a temporary hack on our local (OpenBSD) source tree we made IFAFREE
do nothing. This causes a small memory leak whenever we really do
want to free an ifa, but it also lets me sleep at night. ;)
A proper fix would be to change ifa_refcnt to type long in the ifaddr
structure. A whole bunch of userland stuff will need recompiling too.
>Release-Note:
>Audit-Trail:
>Unformatted:
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-bugs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?19990313153233.844A814D93>