Static Code Analysis

From owner-svn-doc-head@FreeBSD.ORG Mon Sep 9 16:05:59 2013 Return-Path: Delivered-To: svn-doc-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DD6143F3; Mon, 9 Sep 2013 16:05:59 +0000 (UTC) (envelope-from pgj@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C9A112129; Mon, 9 Sep 2013 16:05:59 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id r89G5xkn039420; Mon, 9 Sep 2013 16:05:59 GMT (envelope-from pgj@svn.freebsd.org) Received: (from pgj@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id r89G5xcv039419; Mon, 9 Sep 2013 16:05:59 GMT (envelope-from pgj@svn.freebsd.org) Message-Id: <201309091605.r89G5xcv039419@svn.freebsd.org> From: Gabor Pali Date: Mon, 9 Sep 2013 16:05:59 +0000 (UTC) To: doc-committers@freebsd.org, svn-doc-all@freebsd.org, svn-doc-head@freebsd.org Subject: svn commit: r42634 - head/en_US.ISO8859-1/htdocs/news/status X-SVN-Group: doc-head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-doc-head@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the doc tree for head List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2013 16:06:00 -0000 Author: pgj Date: Mon Sep 9 16:05:59 2013 New Revision: 42634 URL: http://svnweb.freebsd.org/changeset/doc/42634 Log: - Add Q3 reports on static code analysis [1] and CAM locking [2] Submitted by: uqs [1], mav [2] Modified: head/en_US.ISO8859-1/htdocs/news/status/report-2013-07-2013-09.xml Modified: head/en_US.ISO8859-1/htdocs/news/status/report-2013-07-2013-09.xml ============================================================================== --- head/en_US.ISO8859-1/htdocs/news/status/report-2013-07-2013-09.xml Mon Sep 9 14:54:33 2013 (r42633) +++ head/en_US.ISO8859-1/htdocs/news/status/report-2013-07-2013-09.xml Mon Sep 9 16:05:59 2013 (r42634) @@ -19,7 +19,7 @@

Thanks to all the reporters for the excellent work! This report - contains 1 entries and we hope you enjoy reading it.

+ contains 3 entries and we hope you enjoy reading it.

The deadline for submissions covering between October and @@ -27,6 +27,12 @@ + proj + + Projects + + + kern Kernel @@ -60,4 +66,126 @@ framework and crypto(4).

+ + + Static Code Analysis + + + + + Ulrich + Spoerlein + + uqs@FreeBSD.org + + + + + Coverity Scan + Clang Static Analyzer Scan for &os; + Clang Static Analyzer Home Page + + + +

With our own (old and unstable) instance of Coverity Prevent + gone, we have now fully transitioned to the Scan project run by + Coverity (see links), which Open Source projects can use to + learn about possible defects in their source code.

+ +

We also continue to run our code base through the Static + Analyzer that is shipped with Clang/LLVM. It cannot track the + state of the code over time, but has the benefit that everyone + can use it without any special setup. See the home page at the + links section for more information on the Clang Static Analyzer + project in general, and head over to the &os; Clang Static + Analyzer Scan page (see links) to see those possible defects (no + signup required).

+ +

We are looking for a co-admin for both of these projects to + increase the bus-factor and the chance of survival for these + services. Fame and fortune await!

+ + + + Maybe turn on email reports for new defects to the internal + list of &os; developers. + + Find co-admin. + + Fix the defects reported by Coverity and Clang. + + + + + GEOM Direct Dispatch and Fine-Grained CAM Locking + + + + + Alexander + Motin + + mav@FreeBSD.org + + + + + Project SVN branch + Project patches + + + +

Last year's high-performance storage vendors reported + performance bottleneck in &os; block storage subsystem, limiting + peak performance around 300-500K IOPS. While that is still more + then enough for average systems, detailed investigation has + shown number of places that require radical improvement. + Unmapped I/O support implemented early this year already + improved I/O performance by about 30% and moved more accents + toward GEOM and CAM subsystems scalability. Fixing these issues + was the goal of this project.

+ +

The existing GEOM design assumed the most of I/O handling to be + done by only two kernel threads (g_up() and + g_down()). That simplified locking in some cases, but + limited potential SMP scalability and created additional + scheduler overhead. This project introduces concept of direct + I/O dispatch into GEOM for cases where it is know to be safe and + efficient. That implies marking some of GEOM consumers and + providers with one or two new flags, declaring situations when + direct function call can be used instead of normal request + queuing. That allows to avoid any context switches inside GEOM + for the most widely used topologies, simultaneously processing + multiple I/Os from multiple calling threads.

+ +

Having GEOM passing through multiple concurrent calls down to + the underlying layers exposed major lock congestion in CAM. In + existing CAM design all devices connected to the same ATA/SCSI + controller are sharing single lock, which can be quite busy due + to multiple controller hardware accesses and/or code logic. + Experiments have shown that applying only above GEOM direct + dispatch changes burns up to 60% of system CPU time or even more + in attempts to obtain these locks by multiple callers, killing + any benefits of GEOM direct dispatch. To overcome that new + fine-grained CAM locking design was implemented. It implies + splitting big per-SIM locks into several smaller ones: per-LUN + locks, per-bus locks, queue locks, etc. After these changes + remaining per-SIM lock protects only controller driver + internals, reducing lock congestion down to acceptable level and + allowing to keep compatibility with existing drivers.

+ +

Together GEOM and CAM changes twice increase peak I/O rate, + reaching up to 1,000,000 IOPS on contemporary hardware.

+ +

The changes were tested by number of people and are going to be + committed into &os; head and merged to + stable/10 after the end of &os; 10.0 release cycle.

+ +

The project is sponsored by iXsystems, Inc.

+ + + + More reviews, more stability and performance tests. + +