Date: Wed, 29 Dec 2010 03:17:41 -0800 From: Garrett Cooper <gcooper@FreeBSD.org> To: Alexander Leidinger <Alexander@leidinger.net> Cc: "Bjoern A. Zeeb" <bzeeb-lists@lists.zabbadoz.net>, freebsd-arch@freebsd.org Subject: Re: Schedule for releases Message-ID: <AANLkTimKjMoX7O05etAj_-MdWkhQkv1=gD0PjZ-w=xsG@mail.gmail.com> In-Reply-To: <20101229000130.000052f4@unknown> References: <DB4D8AC7-25D6-4901-BBF9-77BEB956840B@cederstrand.dk> <201012220852.oBM8q2Qi039123@lurza.secnetix.de> <20101222123834.GN23098@acme.spoerlein.net> <20101227131140.H6126@maildrop.int.zabbadoz.net> <20101229000130.000052f4@unknown>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Dec 28, 2010 at 3:01 PM, Alexander Leidinger <Alexander@leidinger.net> wrote: ... >> I am still not convinced that whatever development model people and >> companies use (and I heard of in here) is better than to just devel >> on HEAD and if it works there merge it and backport it to your release >> branch for QA and shipping. > > It may not be a problem for developers which know enough about FreeBSD, > but try to sell this to people which do not know enough about FreeBSD > or some management-people (and I'm not talking about the > money-argument here). Bottom line is always feature set in products and ship date from what I've seen, unless someone understands the implications of licensing and takes a look at the big picture as far as maintainability is concerned. Or maybe that little culture is just Cisco and their desire for filling in pretty checkboxes based on what marketing thinks that users want driven by a program management team and managers *shrug*. >From what I see many people like Linux because it's an established name in free software, there are support companies that `reduce' the need to maintain software because they produce the packages, tools, and infrastructure to make your life `easier' (Android SDK, Montavista, etc). and there are N times many code monkeys in the world that `know' how do hack a Linux distro fifteen different ways to Sunday, compared to developers who work within the opensource communities. Because people see a prototype out the door, and can assemble projects faster with OS X.Y.Z, they usually go with that over something else (sometimes BSD, sometimes not). >> We still lack the parts that would tell us something in the last week >> or last 24 hours caused a regression that made my TCP/NFS/ZFS/UFS/<you >> name it> n% slower. =A0Kris had been doing a good job in the past but as >> time shows we need more people, different setups, ... > > We do not lack the parts, we lack someone to take the parts and get > them up and running. See: > =A0http://www.mail-archive.com/freebsd-performance@freebsd.org/msg02819.h= tml > =A0http://www.mail-archive.com/freebsd-performance@freebsd.org/msg02821.h= tml No: according to Erik in the first link, the parts were never fully integrated and the project as a whole was never completed due to lack of hardware and time. Some bits are also just directions as well as Erik says in his reply to the thread. After seeing enough folks prematurely claim victory within my own company, I know enough that nothing is complete unless it works (to some basic degree, i.e. it can be replicated by someone other than the original author of the infrastructure or code using the directions provided from scratch). >> It's not only "compiles", "boots", but also the formerly in this >> thread mentioned "works correctly" and in addition to that the "works >> well as expected" or "works better than before" - hopefully;). > > Maybe this could also be used to run the regression tests as one of the > benchmarks. If yes: As Robert mentioned, we can not go and tell to run > them all in one command (ATM), but we could have each of them as a > different benchmark. Regression tests should be stable. Regression tests shouldn't crash targets (in general). Benchmarks and stress tests can do that. We need regression tests that can be run on a nightly/weekly/monthly/whatever basis with the needed level of breadth and depth on a dedicated set of cluster machines that we can gather results with in a sane manner otherwise whoever is managing the machines will run into the maintenance nightmare that portmgr was/is running into with the tinderbox machines on a regular/periodic basis (ask Mark Linimon about that). ports infrastructure is partly to blame, yes, but a large part of things as well is infrastructure tied up in pointyhat, etc as far as was explained / conveyed to me over email and IRC. As much as I love Yahoo! and the infrastructure work that Yahoo! has provided, a lot of what I see wrong with how things are done with some of the work that Kris did is that it was never made portable, and documentation is lacking in areas -- thus duplicating infrastructure, results, success, and profit is difficult. IMO we should have something like: make checkworld make checkkernel that would go off and run functional regression tests, and something like: make stressworld make stresskernel that would to similar for benchmarks as well. Having stuff tied together between the sources and somewhere in the tree is a minor infrastructure hurdle to get over, but a must for integration. kyua (because ATF has been abandoned as a project except for maintenance releases and bugfixes are concerned) should be integrated in base as well as a tool. Giorgios has engaged Julio from NetBSD about it in the past couple of weeks and I will follow up again as well, time permitting. Adding it to ports is quite frankly a waste of time. That way once the infrastructure is in place, we can easily port over testcases from NetBSD (technically we could do the same right now using the ATF testcases and that might be a good short-term investment even though it's sort of a wasted effort in one respect), and get the coverage that we need on a regular basis to increase confidence in FreeBSD as far as stability on HEAD and MFCs is concerned, so developers can instead focus on writing more code and tests corresponding to that code, so FreeBSD can better scale longterm as a project. Thanks, -Garrett Warning: the above is based on personal opinion as IANAPGM (not a program manager :P), thank my lucky stars...
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTimKjMoX7O05etAj_-MdWkhQkv1=gD0PjZ-w=xsG>