Date: Fri, 24 May 2019 14:42:16 +0000 From: Grzegorz Junka <list1@gjunka.com> To: koobs@FreeBSD.org, freebsd-ports@freebsd.org Subject: Re: Policy on closing bugs Message-ID: <769a6f8f-0785-3683-4a8b-82809904d424@gjunka.com> In-Reply-To: <43a74cfe-5788-3878-026e-a586b6680216@FreeBSD.org> References: <2d6b1503-8ecd-6313-525b-e9f104fcb7f6@gjunka.com> <3ca47a0a-e8ae-e36f-c499-b26f8997e55c@FreeBSD.org> <341fe47b-1104-3050-f85b-504be0460c48@gjunka.com> <43a74cfe-5788-3878-026e-a586b6680216@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 24/05/2019 12:26, Kubilay Kocak wrote: > On 24/05/2019 9:30 pm, Grzegorz Junka wrote: >> >> On 24/05/2019 11:12, Kubilay Kocak wrote: >>> On 24/05/2019 8:07 pm, Grzegorz Junka wrote: >>>> Hey, >>>> >>>> Is there any policy/document when a bug can be closed? For example, >>>> is it OK to close a bug that is fixed upstream but not yet in ports? >>>> >>>> Thanks >>>> GrzegorzJ >>>> >>> >>> Hi Grzegorz, >>> >>> Bugs are closed after they are "resolved". Resolved means a >>> resolution has "occurred", but more precisely, the "thing reported" >>> has been resolved. Resolved doesn't necessary mean "fixed" (see below) >>> >>> What resolution is appropriate/set depends on the context of the >>> issue, usually the specific nature of the request/proposal. Is there >>> a specific bug you're referring to? I can speak to that case >>> specifically if so. >>> >>> For example however, if the bug was a "bug report for the >>> port/package", fixed upstream hasn't fixed the port, so not usually, >>> no, that wouldn't be considered sufficient to be "resolved" and closed. >>> >>> Usually commits upstream are backported to the ports, and they are >>> closed when those are committed. >>> >>> There can't be policies for this perse, as its completely >>> context/request dependent. >>> >>> Resolutions can take place either by way of: >>> >>> 1) A change is made: a commit, usually, but could be a wiki update, >>> or a DNS update for infrastructure changes, etc. >>> 2) One of the 'non-change' resolutions: not accepted, unable to >>> reproduce, feedback timeout, etc >>> >>> Nothing about the above is special or different than most other >>> issue trackers (generally speaking). >>> >>> Regarding states, we have New, Open, In Progress, Closed >>> >>> New: Not touched/Untriaged >>> Open: Initially Triaged (classified) >>> In Progress: Has a real (person) Assignee, action has started >>> Closed: Change(s) Made, OR "Non-Change" resolution set. >>> >>> There's nothing special/different about these either, except that we >>> like to have a real person assigned before in progress, and before >>> close. >>> >>> Happy to answer any more questions regarding issue tracking, etc >>> anytime >>> >> >> Hi Kubilay, >> >> Thank you for a detailed response. Yes, this is related to a >> particular defect. I didn't mention it because I didn't want to be >> picky and seen as causing troubles :) Also wasn't sure what's OK and >> what's not. Here is the defect: > > My pleasure. I understand the desire not to want "cause trouble", but > it's also important that everyone feel comfortable asking questions, > and understand/clarify how things works (or should work, ideally). We > need to see more of it, not less. > >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238086 >> >> I appreciate Yuri's contributions to the community and my intention >> isn't to bring this up for judgment. Even though as a FreeBSD user I >> might feel a bit ignored and shuffled under the carpet after the >> defect has been closed, my intention was more to find out if maybe a >> new state "Postponed" could be added for a defect in a state like >> this one? >> >> GrzegorzJ > > So there's a few issues involved, that are worth making very distinct: > > - A FreeBSD port/package and its users are affected > - Upstream has apparently fixed the issue, but there's not much detail > about how, where, when, etc > - The bugs resolution type > > We'll run through those individually so its hopefully more clear how > they might interact/overlap with each other: > > 1) If a port/package is broken in some way, we want to fix it, and as > maintainers, we have signed up to do that. This is not controversial. > > For users, its not always (I would argue in most cases), possible or > easy for them to distinguish between a port problem, and a software > problem, and who (freebsd or upstream) should be primarily responsible > to a) get the initial bug report b) fix it in the first instance. I > personally don't believe users should be expected to know or do this, > but its great if/when they can. > > There are arguments on both sides of the (unfortunate) > upstream/downstream divide, about users reporting bugs to the "wrong > place". > > Sometimes downstreams hack software to make it work or do things > differently in their distribution/OS, and sometimes these things > break, and upstreams get the report. > > Sometimes upstreams break things, and downstreams get the reports. > > It's a difficult problem to solve completely and permanently, so > ultimately it's the relationship between downstreams/upstreams that is > the most important thing to cultivate. > > Having said that, at least in the former case, I don't think its too > much of a burden for us to receive reports and close them (where > appropriate) as "wont fix / not accepted" after commenting that the > report should go upstream. > > The question as to what and when is appropriate is very case > dependent, but the minimum (in my opinion) is that it should be > explicitly clear to the reporter and documented what the complete > rationale, analysis is to resolving in that manner. > > 2) If an upstream has fixed an issue, all else being equal, we ought > to be motivated to identify the specific upstream fix/commit/pr/issue, > and look to include it in the port, if possible without a version > update. In particular, we should do this so that the fix can be merged > to the quarterly (security and bugfix branch), which doesn't take > version (feature) updates. Bugfixes and security updates is the > promise of quarterly, if we wait for version/feature updates to get > bugfixes, quarterly doesn't get them. > > It's *very* helpful if users can help identify the specific upstream > references, but it's definitely not a requirement. > > Note also that bugs can and should be re-opened by users *at any time* > if additional information comes to hand that precludes or updates the > last 'resolution' at last close. This does not mean that they should > be re-opened spuriously, or because you don't like the decision > personally. > > 3) The resolution in this case "not a bug" is not the most correct for > the apparent resolution "wait for upstream update". It is a bug, and > it has (apparently?) been fixed upstream, and a freebsd port is > currently impacted by it. > > Implicit in a bug report "port foo is broken when bar" is "and the bug > should be fixed in the port". If a user included a patch to fix it, > and the maintainer declined the change, the bug would be closed "not > accepted". > > That there exists some commit upstream that fixed the issue, means the > most correct resolution (of the ones we have today) would be 'Not > Accepted'. Its only slightly not quite "not accepted" because an > upstream commit/fix was not identified (or was, but it wasn't > commented on). Does it mean that strictly speaking my bug was not accepted and has been closed because it didn't offer a fix? I consider bugzilla as a tracking system, not a database of defect statutes. For example, if as an user I see that a port is failing when some specific option is (un)set (like in this case), then first of all I try to raise an issue to see if this problem is known. If I see that the issue has been raised and closed I would expect the option to either: 1. work (meaning the build is successful when it's (un)set as I tried earlier) 2. be removed or otherwise marked as broken In this particular case another user may try to (un)set the option as I did, then waste time on compiling only to discover the build fails. Then if they are inclined enough they may see if this problem is known. It's a step up assuming they are technical enough to look up for a bug, but let's say a fair amount of them may find it in bugzilla. Then they look and see that oh, it's a known problem, but has been closed nevertheless. That's quite inconsiderate for our users and in User Experience is called Dead End: https://signalinc.com/website-ux-dont-leave-users-hanging/ https://uxmag.com/articles/usability-tip-no-dead-ends-please Sure, the user may open the bug and look through the history, read comments, make sense of the reason for closing and infer what would be the next action. It's again a step up in assuming they are technical enough to do that. If the intended audience of FreeBSD and poudriere are developers then it's a fair assumption. Otherwise we are making the system too difficult for newbies to use. IMHO in the case like this (possibly fixed upstream, the date when it's going to land in ports is not known) the most user friendly approach would be to mark the defect as In progress, then remove the broken option or mark it as broken, and in any case remove it from defaults if it's there. Then close the defect with resolution as discussed here. A less friendly option would be to mark the defect as Postponed (if such a state existed). From user experience point of view this would have a number of benefits above closing the defect outright: 1. Users experiencing build problems see straight away that the issue is known but the resolution isn't available yet and won't be for some time. 2. Once the port has been updated, all postponed defects that depend on that update can be either closed or reopened - a change in the defect at that time triggers emails notifying whomever raised the defect about the update and gives them a chance to revisit the defect. They can then either (un)set the option that wasn't previously available or retest the fix. Closing the defect outright without any action gives no benefits to FreeBSD users, only makes the stats look nice. > > Side Note: I recently changed the resolution name "Rejected" to "Not > Accepted" for obvious reasons, though I can now see that this still > needs to be improved, to cover the case where a 'change/patch' hasn't > been offered. I had considered "Not Accepted / WONT FIX", but that was > too long. I'll think about this some more. > > Very very very few bugs are closed "talk to upstream" or "wait for a > version update", and for those that are, its usually very clear that > upstream is the "better" place for the report, or there are other > issues precluding backporting a fix. > > In this specific case, it would be great to have someone identify the > fix concerned upstream, re-open the bug with those links/references, > and explicitly request that they be included in the port. That's > already what happens in the vast majority of cases, even to the extent > of maintainers creating bug reports/PR's on our users behalf. > > Hope that helps GrzegorzJ! > > If you or anyone else is interested in the subject, don't hesitate to > ping me (us, bugmeister) on IRC (#freebsd-bugs / freenode) to talk > more about issue tracking, productivity, problems, improvements, > policies, etc :) > Thanks for the reminder. I keep forgetting about IRC :)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?769a6f8f-0785-3683-4a8b-82809904d424>