Date: Thu, 21 Jul 2005 16:14:52 -0500 From: linimon@lonesome.com (Mark Linimon) To: Robert Watson <rwatson@FreeBSD.org> Cc: 'Marc Olzheim' <marcolz@stack.nl>, Alexey Yakimovich <aiy@ferens.net>, freebsd-stable@FreeBSD.org Subject: Re: Quality of FreeBSD Message-ID: <20050721211452.GA2412@soaustin.net> In-Reply-To: <20050721194500.W9208@fledge.watson.org> References: <200507211803.j6LI34dV005050@ferens.net> <20050721194500.W9208@fledge.watson.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Jul 21, 2005 at 08:00:40PM +0100, Robert Watson wrote: > [original poster wrote:] > >- I completely agree with MikeM - any kind of complex software could be > >tested with right prepared test cases, specially if they are going to be > >reused in the next release; For static problems -- yes. For dynamic problems, such as race conditions, the problem space you are trying to test is many orders of magnitude more complex. This is true of any engineering discipline, but much more so with software engineering due to the immense complexity of the constructed artifacts. > [rwatson again:] > As has been discussed extensively in this thread and other threads, the > FreeBSD development model typically addresses change at the tree HEAD, > where the changes are tested and evaluated, and then they are back-ported. > Some changes are low-risk, and are backported quickly (minor locking > fixes, error handling, etc). Others are higher risk, and are backported > only when they are felt to have received sufficient testing (driver > re-writes, structural changes). Other changes are considered too large > to ever be backported [ ... ] To add to Robert's comments, there was at least one case during the 5.2 cycle where a large backport was made that destabilized the tree for quite some time. This was not due to any lack of diligence on the developer's part; it turned out that the problems were far more subtle and complex than anyone could have reasonably anticipated. Since that time, AFAICT the sentiment has shifted away from large backports. There is always risk in any backport and the risks escalate dramatically the less compartmentalized the changes are. One of the goals for 6.X and beyond is to try to keep changes more compartmentalized; there was simply no way to do such a thing with e.g. SMP and VM changes. At the same time, the sentiment seems to be "let's debug one set of featureset changes all together and then release them as a major release". Of course, backports also require developer time both to do the initial commit and then, more onerously, the followup support. To conclude this thought, the motivation for changing the way FreeBSD is going to do releases going forwards is to try to mitigate such problems: to try to debug, and release, a smaller set of features with new major releases, and more frequently, and with a better-known schedule (every 18 months). > Notice that we'll be supporting the 4.x branch for several years to come. The limiting factor on the 4.X branch is going to be the ports tree more quickly than the base system, particularly for people running desktop installations. The FreeBSD GNOME team has already announced that they are not going to support 4.X by default in the next major GNOME release due this fall. The next major KDE release will probably not work on the 4.X gcc compiler as well IIUC. There are simply an insufficient amount of developer resources to support releases that have different toolchains, include files, and so on. Staying on 4.X indefinitely is not going to be an option at some point in the future, but when, exactly, is difficult to tell right now. It is fair to note, however, that almost no developer attention is being spent on 4.X except for security problems as they are found. Further, the more people we have stay on 4.X, the less people we have testing whichever release we consider the latest "stable" release, and therefore, the less bugs we'll get fixed on that release. One last thought. It always bears repeating that, except for a handful of cases, people who work on FreeBSD are not being paid to do so. Users should always adjust their expectations accordingly. We do our absolute best given the relatively small number of developers that we do have, but we always need more people who are willing to work on regression testing and QA activities. For the companies which view FreeBSD as 'mission-critical' (and we do welcome them!), I challenge them to consider funding development/testing efforts going forwards. (Yes, a number already do, but more would be welcome.) mcl
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20050721211452.GA2412>