Date: Fri, 25 Apr 2014 11:33:31 -0600 From: Chad Perrin <code@apotheon.net> To: freebsd-security@freebsd.org Subject: Re: OpenSSL static analysis, was: De Raadt + FBSD + OpenSSH + hole? Message-ID: <20140425173331.GA8393@glaze.hydra> In-Reply-To: <86zjj9mivi.fsf@nine.des.no> References: <DC2F9726-881B-4D42-879F-61377CA0210D@mac.com> <8783.1398202137@server1.tristatelogic.com> <20140423003400.GA8271@glaze.hydra> <20140423010054.2891E143D098@rock.dv.isc.org> <20140423012206.GB8271@glaze.hydra> <86bnvpoav7.fsf@nine.des.no> <CAG5KPzyTCTbe_vTcP8HDa_KU0agNZQjzVmQ4XnZZjgGFEVnyaQ@mail.gmail.com> <86zjj9mivi.fsf@nine.des.no>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Apr 25, 2014 at 07:14:25PM +0200, Dag-Erling Smørgrav wrote: > Ben Laurie <benl@freebsd.org> writes: > > Dag-Erling Smørgrav <des@des.no> writes: > > > https://en.wikipedia.org/wiki/Halting_problem > > Curious what the halting problem can tell us about finding/fixing bugs? > > Some participants in this thread claim that there is no such thing as a > false positive from a static analyzer. I'm not sure that's true. I think the claims are much more conservative than that. Specifically, I think it has been claimed that many of the kinds of reports you regard as false positives were disputed, thus suggesting that the rate of false positives is substantially *lower* than you claim -- not that it is *zero*. > > A corollary of the halting problem is that it is impossible to write a > program capable to proving or disproving the correctness of all > programs. Hence, static analysis must perforce produce both false > positive and false negative results. The purpose of static analysis > in a compiler is to identify possible optimizations; therefore it must > be conservative, because a false negative may result in incorrect > code; therefore it will produce many false positives. I am not sure this statement, as you have composed it, has much truth in it either. Yes, it seems to be the case that a program cannot be proven correct for some definition of "correct". This does not necessarily mean that an analyzer cannot yield zero false positives. If it produces false negatives, but no false positives, it has still not proven the program correct. Now . . . it may be that it is "impossible" for an analyzer to fail to report false positives, at least sometimes, but your reasoning in the above quoted paragraph does not adequately make that case. In addition to that, nothing about it being incapable of avoiding all false positives implies that it must report "many" false positives. I am inclined to agree that Clang's static analyzer, given the examples provided in this discussion, *technically* produces fewer false positives than you suggest, though it may produce many false "problems" according to what you consider a problem for your projects. I suppose you might reasonably disagree with me if your definition of what constitutes a "positive" differs from mine, but that does not make my opinion wrong -- only different from yours. On the other hand, I am not sure what your definition of a technical true positive would be, as I think I have only seen you refer to cases where you do not consider a given positive (false or true) to be a "real problem". We have now descended into the realm of arguing about how we argue about what we wish to argue, which I suspect will never naturally swing back into the realm of productive discussion. I apologize for my part in that. I will just add this, though: I consider warnings that essentially consist of indications that I have used ambiguous or otherwise poor coding style to be quite useful in ensuring I write maintainable code. Even if the most dire problems are not addressed by the output of the static analyzer, its output can help me clean up my code to make it more readable and maintainable, and this might in turn help me find where more dire problems lurk in the dusty corners of my code. I consider that a win, and as such I appreciate the benefits of Clang's static analyzer as a tool that, coupled with human judgment (flawed though it may be), can help me write cleaner, more maintainable, less buggy, and safer code. Your mileage may vary. -- Chad Perrin [ original content licensed OWL: http://owl.apotheon.org ]
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140425173331.GA8393>