Date: Tue, 21 Jan 2003 20:29:26 -0800 From: Terry Lambert <tlambert2@mindspring.com> To: Narvi <narvi@haldjas.folklore.ee> Cc: hackers@FreeBSD.ORG Subject: Re: verbose device probing ? Message-ID: <3E2E1E26.2CE307C2@mindspring.com> References: <20030122024713.I43637-100000@haldjas.folklore.ee>
next in thread | previous in thread | raw e-mail | index | archive | help
Narvi wrote: > > Priority means "how much does the group value having this > > fixed". > > > > If the problem is that the wi driver panics the kernel when > > Joe-Bob inserts a prototype card that noly he has, the severity > > is 5 (on a scale of 0-5), but the priority on fixing it for the > > project is probably 0. > > If only Joe-Bob or some other very limited set of people have the > card, then the severity of the bug in the *FreeBSD* bug database > should probably not be 5 - orherwise the database will contain a > large amount of bugs that claim to be of high severity but only > ever affect neglible amounts of people. That inserting a prototype > card can cause a crash in one particular driver is not really a > high impact / severity bug as far as FreeBSD users/developers as a > whole is concerned. I intentionally picked my example so that a negligible number of people would have the card (it's a prototype, so, by definition, it has a tiny production run, usually less than you can count without having to take your shoes off). This was to emphasize the difference between "priority", which is assigned by the project, and "severity", which is either assigned by the user, or assigned as a result of a report by a user, on the basis of some objective criteria, e.g.: Severity Objective criteria 5 Esthetic differences between designers and users 4 Failure to expose a feature in UI 3 Loss of functionality, workaround exists 2 Loss of functionality, no workaround exists 1 Loss of customer data So for the Pentium, we have: FPU bug Sev 1 F00F bug Sev 3 Intel byte order Sev 5 Note that this is merely one possible range and domain set, out of many... I'm not insisting on adoption of a particular objective criteria. The only way this works in a self-submission management system is to ask yes/no questions based on escalation criteria, rather than allowing a user to pick from a list... a user will alsways pick the most severe one they feel they can justify, in context, to try to make the problem report "more important" to the people to whom they're reporting the problem. In FreeBSD's case, this is pretty much moot, but it's how user's have been trained to interact with vendors in order to get timely responses out of them, so it's what will happen. > > In any case, if the priority field is to be meaningful at all, > > there must be a group decision (of some kind) on its value. It > > is *not* something that should be set by the person reporting > > the problem. > > This really applies to both fields. It is a question of labelling > and application - for both what the fields and the potential > numerical values for the fields - mean. Some assignemts of meaning > are more useful (according to a metric) than others. Severity has little or nothing to do with whether or not the project will choose to do anything about the problem. It's based on subjective application of a technology in order to cause an ovjectively classifiable result. For example, say you had an editor that could bomb and take out your file; is that a sev 1 bug, according to the criteria? Or... is it sev 3, because you could copy the file before firing up the editor? Or is it a sev 4, because the editor should be renamed and replaced with a shell script that does the copying for you, before invoking the real editor? > > I'll admit that there is some value for newly arrived uncommitted > > resources to a priority field. On the other hand, realize that > > in most cases, if the priority is really that high, trivial things > > will be fixed, and it will leave only non-trivial problems, which > > are not really in the scope of ability of a "newby off the street" > > who is looking for some way to start contributing. For that, you > > almost need a third field "complexity"... which implies analysis > > of problems, which implies scheduling of unschedulable resources, > > etc.. It's a hard problem, with only volunteer resources available; > > I would argue it was intractable, except on an individual basis, in > > fact. > > And complexity need not be always determinably until the problem is > largely fixed either. But it would be of a large relevenace only if > hard rules were associated with the bug categorisations, instead of > being merely indicators of a sort of the state of the codebase. > Knowing how many 'complex' problems there are with a codebase is > normaly less useful information than knowing how many reported > problems have a large impact or need application of otherwise > unallocated resources. Even if there were a system, where each user impacted by a particular problem could "vote" a higher priority to a bug that effected them, all you would end up with is a system where the number of people impacted on each bug was effectively "polled" to assign some figure of merit. But unless/until that figure of merit has an impact on scheduling of resources to deal with the issue, it's pretty much meaningless to do anything like that, because it won't have any real effect on what work gets done, and what work gets left on the table. > At least with FreeBSD the meaning of the fields can be just technical > and doesn't need an additional layer for marketing. The bugs database is, in fact, a "customer facing system". It exists primarily for marketing. It does serve one technical purpose: by making people post problems there, instead of posting them to a mailing list, it makes it much easier to ignore people who raise inconvenient problems on mailing lists. 8-). That's more of a political benefit. The same purpose could be served by demoting the severity of the filed bug, regardless of the filers intended severity, or the objective severity. That's actually a systems purpose: it's a relief valve, which protects the people doing the work from the people consuming it, in order to maintain their participation in the project, which would likely go away, if they were being beat on by end users for code changes, directly. One of the early postings in this thread was about IBM having meetings to deescalate sev 1 bugs so that "products are not be shipped with sev 1 bugs"; this weasels around an objective quality criteria that is being enforced by release engineers. That's the real intent of such meetings: to work around quality criteria, and ship a lower quality product, so as to meet objective performance criteria, relative to scheduled vs. actual ship date. The difference in the IBM case is that IBM is, effectively, paying people to work around the objective criteria system like this, by basing pay on performance relative to schedules. It doesn't take someone with a profound understanding of games theory to guess at the result: "You get what you pay for". 8-). The problem with applying this sort of thing to volunteer efforts is that you *can't* "just hire someone else". There is no such thing as "an unallocated resource": there's only "self allocated resources". I submit that the severity in the FreeBSD bugs database is far from objectively assigned, and there is no process in place for it to be revisited, once the severity is set, except for arbitrary changes made by people who don't want to have the project be seen with a large number of sever bugs, and therfore have a vested interest in underestimating severity (i.e. "the right answer" on the editor, earlier, is that it's a sev 3, and a user who just lost six hours of work on their Master's Thesis since their last exit/entry would likely consider it a sev 1, and someone who'd rather codify the workaround, rather than fix the root cause, would write the script, mark it a sev 2, and close it). Priority... priority is totally meaningless, unless you are willing to accept stall barriers, either in terms of resource scheduling, or in terms of resource levelling across available problems (the second is less likely, since software engineering is very hard to resource-level, even if the project managers who have found that menu button in "Microsoft Project" would wish otherwise, since not all engineers know the VM system or the serial drivers or the EMACS LISP interpreter equally well). Short of not letting Joe-Bob check in code against a sev 4 while he has a sev 2 outstanding that he "owns" -- *and* assigning ownership of bugs, so it's his bug, and not an unassigned one -- there's really very little you can do in terms of scheduling. Even so, many people will just say "screw you", and not work on anything (they are volunteers... it's not like you could fire them and deprive them of a paycheck as a punative measure), rather than work on something they don't want to work on. So it still comes down to enforcement, and you don't have enforcement available. The closest you can possibly come is "peer pressure", and that only works if the group assigning the priorities is generally well respected by the volunteers. So it's not like you can just "declare process" and have it fix things, either (that only works if you have a stick to go with the carrot that got people to do the work in the first place). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3E2E1E26.2CE307C2>