Date: Thu, 29 Aug 2019 17:09:32 +0200 From: "Kristof Provost" <kp@FreeBSD.org> To: araujo@freebsd.org Cc: "Ian Lepore" <ian@freebsd.org>, "Konstantin Belousov" <kostikbel@gmail.com>, "FreeBSD Hackers" <freebsd-hackers@freebsd.org>, "Li-Wen Hsu" <lwhsu@freebsd.org>, fcp@freebsd.org Subject: Re: FCP 20190401-ci_policy: CI policy Message-ID: <DF8CB241-54AE-49DE-88F2-F28189486ED2@FreeBSD.org> In-Reply-To: <CAOfEmZgEbT7ni80vWehHm%2B4oPyH3m%2Brb0M_VyxHmNM3rkhyG1Q@mail.gmail.com> References: <CAKBkRUwKKPKwRvUs00ja0%2BG9vCBB1pKhv6zBS-F-hb=pqMzSxQ@mail.gmail.com> <20190829114057.GZ71821@kib.kiev.ua> <28934eb780342605090bf365ac3a2e0d522256f5.camel@freebsd.org> <A837EF78-DC69-4B52-A7D9-0363302A48FA@FreeBSD.org> <CAOfEmZgEbT7ni80vWehHm%2B4oPyH3m%2Brb0M_VyxHmNM3rkhyG1Q@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 29 Aug 2019, at 17:01, Marcelo Araujo wrote: > Em qui, 29 de ago de 2019 =C3=A0s 22:54, Kristof Provost <kp@freebsd.or= g> > escreveu: > >> On 29 Aug 2019, at 16:42, Ian Lepore wrote: >>> (And I don't think breaking a test counts as >>> breaking the build.) >>> >> I fundamentally disagree on this point. A test failure is, just like = >> a >> compiler warning, a precious gift that should not be ignored. >> The more distance (both in terms of time, and in terms of the people >> involved) there is between a bug being introduced and it being = >> detected >> the harder it is to fix it. Test accelerate detection of bugs. If we = >> do >> not take test failures seriously (i.e. as an indication something is >> wrong and should be fixed) the tests will inevitable become useless = >> in >> one of two ways: we=E2=80=99ll either disable failing tests (which is = what = >> we >> tend to do now) reducing test coverage or we=E2=80=99ll have a test su= ite = >> with >> many failures in it, which makes it useless as well. (As with = >> compiler >> warnings, the best way to keep them under control is to consider them = >> to >> be fatal errors.) >> > > Could you elaborate where is the "fundamentally" you disagree? Where = > is the > fundament? You guys are introducing something new, yes everybody knows > about test, it is year 2019, but nobody can come with new rules tha in > hours we gonna revert if you "dare to don't fix it". Sorry, this is = > not how > people test software and fix it. > I do think that breaking a test breaks the build. Something used to work = and now it doesn=E2=80=99t. That=E2=80=99s breakage, even if it=E2=80=99s= not as total as = it not compiling any more. >> In either scenario we end up reducing test coverage, which means = >> we=E2=80=99re >> going to push more bugs towards users. >> >>> I totally agree. This is an overly-bureaucratic solution in search = >>> of >>> a problem. >>> >>> If this needs to be addressed at all (and I'm not sure it does), = >>> then >>> another sentence or two in bullet item 10 in section 18.1 [*] of the >>> committer's guide should be enough. And even then it needn't be >>> overly-formal and should just mention that if a commit does break = >>> the >>> build the committer is expected to be responsive to that problem and >>> the commit might get reverted if they're unresponsive. I don't = >>> think >>> we need schedules. >>> >> I do feel that=E2=80=99s a better argument. We=E2=80=99ve always had a= policy of >> reverting on request (AIUI), so this is more or less trying to be a >> strong restatement of that, more than a fundamental shift in policy. >> > > We don't have a policy to revert commit, actually revert commit is > something bad, it is kind of punishment, I have been there, nobody = > wants to > be there. Stop to push this non-sense argument. > That=E2=80=99s how I=E2=80=99ve interpreted =E2=80=9911. Developer Relati= ons=E2=80=99 in in = https://www.freebsd.org/doc/en_US.ISO8859-1/articles/committers-guide/art= icle.html#conventions = (Specifically, =E2=80=9C If a commit does results in controversy erupti= ng, = it may be advisable to consider backing the change out again until the = matter is settled.=E2=80=9D) I understand that it=E2=80=99s not fun to see changes reverted, and it=E2= =80=99s = certainly not the intention to make that the preferred solution. = That=E2=80=99s why the FCP discusses adds waiting periods and discussion = with = the committer. Best regards, Kristof
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?DF8CB241-54AE-49DE-88F2-F28189486ED2>