From owner-freebsd-net@FreeBSD.ORG Thu Feb 14 21:55:53 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D88477D1 for ; Thu, 14 Feb 2013 21:55:53 +0000 (UTC) (envelope-from bharavi_oak@yahoo.com) Received: from nm22-vm3.bullet.mail.gq1.yahoo.com (nm22-vm3.bullet.mail.gq1.yahoo.com [98.136.217.66]) by mx1.freebsd.org (Postfix) with ESMTP id 9F358FDB for ; Thu, 14 Feb 2013 21:55:53 +0000 (UTC) Received: from [98.137.12.62] by nm22.bullet.mail.gq1.yahoo.com with NNFMP; 14 Feb 2013 21:55:53 -0000 Received: from [98.137.12.212] by tm7.bullet.mail.gq1.yahoo.com with NNFMP; 14 Feb 2013 21:55:53 -0000 Received: from [127.0.0.1] by omp1020.mail.gq1.yahoo.com with NNFMP; 14 Feb 2013 21:55:53 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 42375.42813.bm@omp1020.mail.gq1.yahoo.com Received: (qmail 39815 invoked by uid 60001); 14 Feb 2013 21:55:52 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1360878952; bh=KqRtB/itRQP9Bo2waRcG9AlZjnX7ZA2qwfM+MIR485g=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:MIME-Version:Content-Type; b=2crAq4jRGny48M/FH6lZFPh5/ZxrZxL1+9HA9enUpOIyQPbdGN3anGzdDXCN79bOSMrScIl+VKmEgyMdekhCxwWcIEI5MliKYH9uBzEFFzv4KnYSC2pYVtNwm/X9IflPicPhQbU1rOwAYqddpVSxYgRKYgI+VRH8PQybLXQ/Oo0= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:MIME-Version:Content-Type; b=WjfZUwGlebzuyoxJOZXr+rVwnbXBDQjV1ltFncJUSio2shPsmyNmnFnhAuUR5umR67/8oFqrZD5flZM92ERhdc3Gvny2NqSQqXWeeJrBXJ6JlLrUUGQEn1ml1MvJ4hjhp5hryqEbmbLej9m7tV5WwvHhN5gVWNQSqTUo8EYerdI=; X-YMail-OSG: KtjumQ4VM1lOWIph_NLNl7pxoLBeFT3Pb4q6XnB0vMl_S8N RUoTZTvkonKoGa0m.NCyxKBSplJRhSHAIN_4TgQgR_PcJLWvK4iXG7u2eBsp a8UbaPWoaTKd0kKbrDMinBySyyfg_nS18MBsT8b46p1k2NI4xFshw79XBhqP BuKvAtPpRs0aVmT7JhBQoPmNDAXo9RRx2D9tamZlDMGziVX5q5GIInrX2kMX QGGePNM5HtAR6IhAlcnloSaiyI7S2kTpL4atPOEzsf5pGtyolk1sk3aaxkAs Q_RFfgoGZPQxZuhV2zlTodXfdEXmEOqfzfCU.Ex4dk4FHGUS.zP7Wcfn3yff jwOZArt.EbPVWlQJGWdTiRJor9uh89iNAANRlIc.BM8LlBoZTs6Z7exL_Cjq 5ZJrTRGytd940Lr0fbTYoFHXMb2y2TC43WUQXislH4GoZNtFZjkxLwntlvDt 5UoGPkxv7pTAIh34VhKzUgF1bYv0YZCLcPOL0dj3eedUTE31hkcWH_LYwUYF 6TJU- Received: from [137.69.117.57] by web164604.mail.gq1.yahoo.com via HTTP; Thu, 14 Feb 2013 13:55:52 PST X-Rocket-MIMEInfo: 001.001, SGksDQoNCkluIHZsYW5fdW5jb25maWdfbG9ja2VkLCB2bGFuIHRydW5rIGlzIGRlc3Ryb3llZCB2aWEgdHJ1bmtfZGVzdHJveSBmdW5jdGlvbiB3aGljaCBmcmVlcyBoYXNoLCBkZXN0cm95cyB0cnVuayBsb2NrIGFuZCB0aGVuIGZyZWVzIHRydW5rLiBCZWZvcmUgdHJ1bmtfZGVzdHJveSBpcyBjYWxsZWQsIHRydW5rIGxvY2sgaXMgcmVsZWFzZWQgc28gdGhhdCBhIHRocmVhZCBpbiB2bGFuX2lucHV0LCBpZiBhbnksIGNhbiBmaW5pc2ggaXRzIHdvcmsgKGFzIHBlciB0aGUgY29tbWVudCBpbiB0aGUgY29kZSkBMAEBAQE- X-Mailer: YahooMailClassic/15.1.2 YahooMailWebService/0.8.133.508 Message-ID: <1360878952.34332.YahooMailClassic@web164604.mail.gq1.yahoo.com> Date: Thu, 14 Feb 2013 13:55:52 -0800 (PST) From: Bharavi Oak Subject: vlan trunk structure race condition between vlan unconfig and vlan_input To: freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Feb 2013 21:55:53 -0000 Hi, In vlan_unconfig_locked, vlan trunk is destroyed via trunk_destroy function which frees hash, destroys trunk lock and then frees trunk. Before trunk_destroy is called, trunk lock is released so that a thread in vlan_input, if any, can finish its work (as per the comment in the code). However, we have encountered a condition wherein this release of lock was not enough for the other thread to grab the lock and finish its work. Instead, what happened was that this other thread (doing vlan_input) got the lock when this thread (destroying vlan) had already freed hash inside trunk_destroy. As a result of this, the vlan_input thread got NULL hash (in vlan_gethash) causing panic. In other words, execution of a thread in vlan_input could reach TRUNK_RLOCK when another thread doing vlan unconfig is at any point in its execution; it can be before trunk_destroy or at any point within trunk_destroy. In our case, it caused trunk->hash to be null while trunk was still not null. But it may as well happen that the trunk itself has been freed when the first thread reaches TRUNK_RLOCK. Any way, both cases would cause panic. When trying to solve this, we have started working on the following lines: Bring trunk lock outside of ifvlantrunk structure keeping intact the pointers to it in ifvlan and ifnet structures. So, the reference to trunk lock would not depend on validity of trunk. In trunk destroy, delay freeing hash as much as possible, perhaps by checking if any read waiters are present. However, this would also require any thread doing vlan_input to reach TRUNK_RLOCK latest by this instance. We may also check for validity of hash inside vlan_gethash; but this would add some extra instructions that are not required otherwise. So, can a FreeBSD developer please comment on this problem and the possible way to solve this as mentioned above. Thanks, Bharavi Note that although we encountered this in FreeBSD 7.x, there is no difference in this part of code in FreeBSD 9.x as well.