From owner-freebsd-security@FreeBSD.ORG Fri Apr 25 17:33:44 2014 Return-Path: Delivered-To: freebsd-security@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C0715DCB for ; Fri, 25 Apr 2014 17:33:44 +0000 (UTC) Received: from gproxy1-pub.mail.unifiedlayer.com (gproxy1-pub.mail.unifiedlayer.com [69.89.25.95]) by mx1.freebsd.org (Postfix) with SMTP id 8A0301084 for ; Fri, 25 Apr 2014 17:33:44 +0000 (UTC) Received: (qmail 14984 invoked by uid 0); 25 Apr 2014 17:33:43 -0000 Received: from unknown (HELO CMOut01) (10.0.90.82) by gproxy1.mail.unifiedlayer.com with SMTP; 25 Apr 2014 17:33:43 -0000 Received: from box543.bluehost.com ([74.220.219.143]) by CMOut01 with id uHZc1n00C36DqkS01HZfdU; Fri, 25 Apr 2014 11:33:43 -0600 X-Authority-Analysis: v=2.1 cv=EOmVjTpC c=1 sm=1 tr=0 a=m1eD20qHdBbyQr3wvGb0tQ==:117 a=m1eD20qHdBbyQr3wvGb0tQ==:17 a=cNaOj0WVAAAA:8 a=f5113yIGAAAA:8 a=J0QyKEt1u0cA:10 a=oA0Jii2iB0UA:10 a=ZzjhlJrv0foA:10 a=8nJEP1OIZ-IA:10 a=hBmbxFWgAAAA:8 a=O5JQB85wRqYA:10 a=9NnC__TRAO0A:10 a=6I5d2MoRAAAA:8 a=8pif782wAAAA:8 a=70qzlKQjAAAA:8 a=1G-YfAeunkake7ZTHKgA:9 a=nGmemLi6sPGI8RlX:21 a=0Uqwa3fj9EtE_n6K:21 a=wPNLvfGTeEIA:10 a=SV7veod9ZcQA:10 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=apotheon.net; s=default; h=In-Reply-To:Content-Transfer-Encoding:Content-Type:MIME-Version:References:Message-ID:Subject:To:From:Date; bh=kIn6dPSI4g1o46ZDQtu/a25jY4Y+7qMIzVoB1392PdY=; b=MFYtz59PQhT1Yr76sUhV/qpGIKLolOXF/Fk4MOkSYtPqoL+F9a9TgSCmOUMLrKGYdNmXyPtOEh4Hjbw8hiLvKA2uTA0J/d3iM6pot2a7TvFd2LHz9zTtRCTBAGibWH+q; Received: from [98.245.97.34] (port=62459 helo=localhost) by box543.bluehost.com with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82) (envelope-from ) id 1Wdk0G-0008Px-2i for freebsd-security@freebsd.org; Fri, 25 Apr 2014 11:33:36 -0600 Date: Fri, 25 Apr 2014 11:33:31 -0600 From: Chad Perrin To: freebsd-security@freebsd.org Subject: Re: OpenSSL static analysis, was: De Raadt + FBSD + OpenSSH + hole? Message-ID: <20140425173331.GA8393@glaze.hydra> Mail-Followup-To: freebsd-security@freebsd.org References: <8783.1398202137@server1.tristatelogic.com> <20140423003400.GA8271@glaze.hydra> <20140423010054.2891E143D098@rock.dv.isc.org> <20140423012206.GB8271@glaze.hydra> <86bnvpoav7.fsf@nine.des.no> <86zjj9mivi.fsf@nine.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <86zjj9mivi.fsf@nine.des.no> User-Agent: Mutt/1.5.23 (2014-03-12) X-Identified-User: {2737:box543.bluehost.com:apotheon:apotheon.net} {sentby:smtp auth 98.245.97.34 authed with code@apotheon.net} X-BeenThere: freebsd-security@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Security issues \[members-only posting\]" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 17:33:44 -0000 On Fri, Apr 25, 2014 at 07:14:25PM +0200, Dag-Erling Smørgrav wrote: > Ben Laurie writes: > > Dag-Erling Smørgrav writes: > > > https://en.wikipedia.org/wiki/Halting_problem > > Curious what the halting problem can tell us about finding/fixing bugs? > > Some participants in this thread claim that there is no such thing as a > false positive from a static analyzer. I'm not sure that's true. I think the claims are much more conservative than that. Specifically, I think it has been claimed that many of the kinds of reports you regard as false positives were disputed, thus suggesting that the rate of false positives is substantially *lower* than you claim -- not that it is *zero*. > > A corollary of the halting problem is that it is impossible to write a > program capable to proving or disproving the correctness of all > programs. Hence, static analysis must perforce produce both false > positive and false negative results. The purpose of static analysis > in a compiler is to identify possible optimizations; therefore it must > be conservative, because a false negative may result in incorrect > code; therefore it will produce many false positives. I am not sure this statement, as you have composed it, has much truth in it either. Yes, it seems to be the case that a program cannot be proven correct for some definition of "correct". This does not necessarily mean that an analyzer cannot yield zero false positives. If it produces false negatives, but no false positives, it has still not proven the program correct. Now . . . it may be that it is "impossible" for an analyzer to fail to report false positives, at least sometimes, but your reasoning in the above quoted paragraph does not adequately make that case. In addition to that, nothing about it being incapable of avoiding all false positives implies that it must report "many" false positives. I am inclined to agree that Clang's static analyzer, given the examples provided in this discussion, *technically* produces fewer false positives than you suggest, though it may produce many false "problems" according to what you consider a problem for your projects. I suppose you might reasonably disagree with me if your definition of what constitutes a "positive" differs from mine, but that does not make my opinion wrong -- only different from yours. On the other hand, I am not sure what your definition of a technical true positive would be, as I think I have only seen you refer to cases where you do not consider a given positive (false or true) to be a "real problem". We have now descended into the realm of arguing about how we argue about what we wish to argue, which I suspect will never naturally swing back into the realm of productive discussion. I apologize for my part in that. I will just add this, though: I consider warnings that essentially consist of indications that I have used ambiguous or otherwise poor coding style to be quite useful in ensuring I write maintainable code. Even if the most dire problems are not addressed by the output of the static analyzer, its output can help me clean up my code to make it more readable and maintainable, and this might in turn help me find where more dire problems lurk in the dusty corners of my code. I consider that a win, and as such I appreciate the benefits of Clang's static analyzer as a tool that, coupled with human judgment (flawed though it may be), can help me write cleaner, more maintainable, less buggy, and safer code. Your mileage may vary. -- Chad Perrin [ original content licensed OWL: http://owl.apotheon.org ]