From owner-freebsd-hackers@freebsd.org Wed Jul 11 18:22:16 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4BB81031C7C for ; Wed, 11 Jul 2018 18:22:16 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 68E707F901 for ; Wed, 11 Jul 2018 18:22:16 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from [192.168.200.3] (c-73-216-227-39.hsd1.va.comcast.net [73.216.227.39]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: gallatin) by duke.cs.duke.edu (Postfix) with ESMTPSA id 9F7A527000A0; Wed, 11 Jul 2018 14:22:08 -0400 (EDT) DMARC-Filter: OpenDMARC Filter v1.3.1 duke.cs.duke.edu 9F7A527000A0 Authentication-Results: duke.cs.duke.edu; dmarc=none header.from=cs.duke.edu DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=cs.duke.edu; s=mail0816; t=1531333328; bh=RvgsTVUfmthwWFRI37ZnzqGmGa4JCFotswKI9A2N5xg=; h=Subject:To:From:Date:From; b=FbSAk7GfU40ttTEoSoWTygKcd1OTK3aj9XYaA2dYZ6lfVwMfhR21XANNrIS8xCie7 gtDgSR4XS+b5m2SaREHjCyAtG4Jws2jzYF7zCv8AZLg2zGpfIVGtl6U+dyzm2k8o/w acLNpykjHKtuzukO7riUJRVwuzZ/Sfy8K8E2AUmO9tqhAqImIHrwh+awZ1HnUsjsno b8tHoJOrwlI8E4L1gp7eiR+TJmLbnUElxexYQDsforJuSx9NYSd0R/iAqfPHGqd0fr xqxNpzcqeDGrgc0raDlgS4yDL62eiTFgVS+D26E/wGrPgvQD82/IVAOfyy8lcJAiB0 YbdNkj+YHU7ZQ== Subject: Re: high CPU usage in FreeBSD for a PCIe card driver To: Steevan Rodrigues Cc: freebsd-hackers@freebsd.org References: <2ae78782-6cca-452e-d004-3999885ae4e0@cs.duke.edu> From: Andrew Gallatin Message-ID: <03274bb2-87b1-ab32-ee15-81e83b6f6bdd@cs.duke.edu> Date: Wed, 11 Jul 2018 14:22:07 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jul 2018 18:22:16 -0000 On 07/11/18 07:46, Steevan Rodrigues wrote: > Hello Andrew, > > Sorry to bother you.  I am working on this problem again after a break > of few days. > I ran following command  to get lock statistics  when I run my userspace > application on a 12 core 24 thread server PC. > From this data below  to looks like my driver is causing a contention > on a kernel lock  (pmap ).  Am I right ? > >  lockstat -x aggsize=4m -D 20 sleep 10 > > Adaptive mutex spin: 1122679 events in 10.013 seconds (112121 events/sec) > > Count indv cuml rcnt     nsec Lock                   Caller > ------------------------------------------------------------------------------- > 1089662  97%  97% 0.00    65375 pmap                   pmap_extract+0x1d2 > 31805   3% 100% 0.00    14881 cdev                   devvn_refthread+0x142 Yes. You can get more information if you use the -s 10 argument to lockstat. That way, you'll see what's calling pmap_extract. Are you doing frequent device ioctls? Drew