From owner-freebsd-bugs Sat Mar 13 7:40:36 1999 Delivered-To: freebsd-bugs@freebsd.org Received: from freefall.freebsd.org (freefall.FreeBSD.ORG [204.216.27.21]) by hub.freebsd.org (Postfix) with ESMTP id D582614E7F for ; Sat, 13 Mar 1999 07:40:13 -0800 (PST) (envelope-from gnats@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.9.2/8.9.2) id HAA84825; Sat, 13 Mar 1999 07:40:00 -0800 (PST) (envelope-from gnats@FreeBSD.org) Received: by hub.freebsd.org (Postfix, from userid 32767) id 844A814D93; Sat, 13 Mar 1999 07:32:33 -0800 (PST) Message-Id: <19990313153233.844A814D93@hub.freebsd.org> Date: Sat, 13 Mar 1999 07:32:33 -0800 (PST) From: andrewb@demon.net To: freebsd-gnats-submit@freebsd.org X-Send-Pr-Version: www-1.0 Subject: kern/10570: Route delete with many routes on single interface causes panic Sender: owner-freebsd-bugs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >Number: 10570 >Category: kern >Synopsis: Route delete with many routes on single interface causes panic >Confidential: no >Severity: serious >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Sat Mar 13 07:40:00 PST 1999 >Closed-Date: >Last-Modified: >Originator: Andrew Bangs >Release: all >Organization: Demon Internet >Environment: >Description: We've come across a problem that may affect people using *BSD boxes as routers, particularly if they have a large route table where most of the routes go via a single interface. We found this with OpenBSD, but it looks to be a bit of common code going back to the Dark Ages. The ifaddr structure (sys/net/if.h in NetBSD and OpenBSD; sys/net/if_var.h in FreeBSD) looks something like this (this one from OpenBSD) : /* * The ifaddr structure contains information about one address * of an interface. They are maintained by the different address families, * are allocated and attached when an address is set, and are linked * together so all addresses for an interface can be located. */ struct ifaddr { struct sockaddr *ifa_addr; /* address of interface */ struct sockaddr *ifa_dstaddr; /* other end of p-to-p link */ #define ifa_broadaddr ifa_dstaddr /* broadcast address interface */ struct sockaddr *ifa_netmask; /* used to determine subnet */ struct ifnet *ifa_ifp; /* back-pointer to interface */ TAILQ_ENTRY(ifaddr) ifa_list; /* list of addresses for interface */ void (*ifa_rtrequest) /* check or clean routes (+ or -)'d */ __P((int, struct rtentry *, struct sockaddr *)); u_short ifa_flags; /* mostly rt_flags for cloning */ short ifa_refcnt; /* count of references */ int ifa_metric; /* cost of going out this interface */ }; See that 'short ifa_refcnt;' ? How about making that a 'long ifa_refcnt;'? Where people are using *BSD boxes as Internet routers, there are around 56000 routes in a 'full route table', plus however many might be used within an organisation's own network. We're doing some particularly odd things, which leads to having around 75000 routes on some boxes. Even removing the 'odd things' that we're doing with routes we would still be looking at around 60000 (and this number is steadily growing). The total number of routes isn't the problem, but if the number of routes on a single interface goes above 2^16 and later falls below 2^16 then the reference counter hits zero and we free (IFAFREE in if.h and ifafree() in route.c) the ifa. Subsequent route deletions cause a panic was we flail around looking for a reference counter to decrement in a structure that we've just freed (at least, that's the behaviour in OpenBSD, and the code elsewhere looks the same). I'm sending this to the Free, Net and OpenBSD communities in the hope that the problem can be addressed before it bites anyone else. >How-To-Repeat: Add 2^16 routes all with destinations via the same interface. Add another couple of routes with destination via the same interface. Start deleting routes. Watch it panic. (We were using GateD rather than adding these manually.) >Fix: As a temporary hack on our local (OpenBSD) source tree we made IFAFREE do nothing. This causes a small memory leak whenever we really do want to free an ifa, but it also lets me sleep at night. ;) A proper fix would be to change ifa_refcnt to type long in the ifaddr structure. A whole bunch of userland stuff will need recompiling too. >Release-Note: >Audit-Trail: >Unformatted: To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-bugs" in the body of the message