From owner-freebsd-transport@freebsd.org Wed Jul 20 19:22:23 2016 Return-Path: Delivered-To: freebsd-transport@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5FA54B9E132 for ; Wed, 20 Jul 2016 19:22:23 +0000 (UTC) (envelope-from gallatin@netflix.com) Received: from mail-pa0-x232.google.com (mail-pa0-x232.google.com [IPv6:2607:f8b0:400e:c03::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2F54F1660 for ; Wed, 20 Jul 2016 19:22:23 +0000 (UTC) (envelope-from gallatin@netflix.com) Received: by mail-pa0-x232.google.com with SMTP id ks6so21072095pab.0 for ; Wed, 20 Jul 2016 12:22:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netflix.com; s=google; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=3puGC85BXxhFT0/pHgJyJgiPr7g7SvkCRnkY1D61G2g=; b=f+SM1QWH1rTowvfu5uZsUcaZl5dgR8KUdGMlCz+X32KKW0nJMMjYQYYVOiQQi36dmk oxDRcS8NDlC/KYymuE2ygj0VEspeXG+RDLqF560PTjMMA0orlKJ46Dz+rgVJRdRhTACB 1OV7uKYrZonBs8p/Gq7opB2V+cDzn0QV8LRGI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=3puGC85BXxhFT0/pHgJyJgiPr7g7SvkCRnkY1D61G2g=; b=GVJ+Ow8jp8+e1xMMgpfeAi0Th614781eU+uJx4oAmnJ5FX/yakp0J/yFiIFtgV0cGp VMqxn13Daov5SYZnR/qOTr8c5pVIKPJuP1ebHwtsFYgveffNWAgxtu4DTylKEKOwGti7 jtLdqXEcs+9IIeEhRBspM7kIgnz+JptekYUD/S0mHuYVeqez+uuKrerdyJ+qGXE2zafr gUZM+LuuwZMyPHwXRXfqzqFPUK0FUFkecz8PMllj7V7+qqcrZpoI1fV/43RGIWNPnPOb GY6Rj37kXC2wSagtBKB+2CQy7KRYCcTjQW4ulvsc9JSh2t7Gj8WeqVuDS1YPo3bsWCak VQqg== X-Gm-Message-State: ALyK8tLjI9QvJvhgngxNYz3tHtcpJDAO5aQ4nHlDKKBi+M/ycFOndVwWNaNirPsFs2JKZsYi X-Received: by 10.66.242.201 with SMTP id ws9mr77017243pac.7.1469042542477; Wed, 20 Jul 2016 12:22:22 -0700 (PDT) Received: from [192.168.200.3] (c-73-147-115-187.hsd1.va.comcast.net. [73.147.115.187]) by smtp.gmail.com with ESMTPSA id h66sm6432208pfe.58.2016.07.20.12.22.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 20 Jul 2016 12:22:21 -0700 (PDT) Subject: Re: in_broadcast() called for almost every packet in ip_output() To: Ryan Stone References: Cc: freebsd-transport@freebsd.org From: Andrew Gallatin Message-ID: <559f1d0e-9d35-f458-bff5-b69a03297cf0@netflix.com> Date: Wed, 20 Jul 2016 15:22:20 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-transport@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussions of transport level network protocols in FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Jul 2016 19:22:23 -0000 On 07/20/2016 15:09, Ryan Stone wrote: > On Wed, Jul 20, 2016 at 3:00 PM, Drew Gallatin > wrote: > > I'd certainly prefer not to add any overhead. Properly managing > this (and other similar) lists is a job for some lightweight > lifecycle based mechanism like concurrency kit or rcu. > > Unless you have a solid reason to fix it, I'd suggest just adding > the locking commented out (the way rwatson did with IN_IFADDR_RLOCK > in ip_input(), so that the next person to trip over it will know > what's going on. > > Drew > > > I'm not clear on the details, but we have some internal tests that were > provoking this exact crash after a couple of days of stress testing. > > I believe that the actual fix would involve caching the result in the > pcb. When the endpoint is already known, there's no reason to check for > a broadcast packet on every single packet. On a system that uses a > large number of IP addresses on an interface (e.g. for jails) the > overhead of just iterating over the list is going to add up no matter > what synchronization mechanism you use. Caching sounds reasonable, but.. won't the lock still add overhead to paths like unconnected udp and packet forwarding?