Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 15 Apr 2020 15:34:12 +0200
From:      "Kristof Provost" <kp@FreeBSD.org>
To:        freebsd-testing@freebsd.org
Cc:        freebsd-current@freebsd.org, freebsd-stable@freebsd.org
Subject:   Re: FreeBSD CI Weekly Report 2020-04-12
Message-ID:  <A2DF53A3-FD86-427D-B1EE-508228B0F4CE@FreeBSD.org>
In-Reply-To: <20200414223710.GB33328@freefall.freebsd.org>
References:  <20200414223710.GB33328@freefall.freebsd.org>

next in thread | previous in thread | raw e-mail | index | archive | help
On 15 Apr 2020, at 0:37, Li-Wen Hsu wrote:
> (Please send the followup to freebsd-testing@ and note Reply-To is 
> set.)
>
> FreeBSD CI Weekly Report 2020-04-12
> ===================================
>
> Here is a summary of the FreeBSD Continuous Integration results for 
> the period
> from 2020-04-06 to 2020-04-12.
>
> During this period, we have:
>
> * 1801 builds (94.0% (+0.4) passed, 6.0% (-0.4) failed) of buildworld 
> and
>   buildkernel (GENERIC and LINT) were executed on aarch64, amd64, 
> armv6,
>   armv7, i386, mips, mips64, powerpc, powerpc64, powerpcspe, riscv64,
>   sparc64 architectures for head, stable/12, stable/11 branches.
> * 288 test runs (25.1% (-24.6) passed, 29.9% (+10.6) unstable, 45.1% 
> (+14.1)
>   exception) were executed on amd64, i386, riscv64 architectures for 
> head,
>   stable/12, stable/11 branches.
> * 30 doc and www builds (83.3% (-1.3) passed, 16.7% (+1.3) failed)
>
> Test case status (on 2020-04-12 23:59):
> | Branch/Architecture | Total     | Pass       | Fail     | Skipped  |
> | ------------------- | --------- | ---------- | -------- | -------- |
> | head/amd64          | 7744 (+4) | 7638 (+19) | 14 (+5)  | 92 (-20) |
> | head/i386           | 7742 (+4) | 7628 (+15) | 16 (+5)  | 98 (-16) |
> | 12-STABLE/amd64     | 7508 (0)  | 7449 (-3)  | 1 (+1)   | 58 (+2)  |
> | 12-STABLE/i386      | 7506 (0)  | 7425 (-17) | 2 (+2)   | 79 (+15) |
> | 11-STABLE/amd64     | 6882 (0)  | 6829 (-6)  | 1 (+1)   | 52 (+5)  |
> | 11-STABLE/i386      | 6880 (0)  | 6749 (-82) | 80 (+80) | 51 (+2)  |
>
> (The statistics from experimental jobs are omitted)
>
> If any of the issues found by CI are in your area of interest or 
> expertise
> please investigate the PRs listed below.
>
> The latest web version of this report is available at
> https://hackmd.io/@FreeBSD-CI/report-20200412 and archive is available 
> at
> https://hackmd.io/@FreeBSD-CI/ , any help is welcome.
>
> ## News
>
> * The test env now loads the required module for firewall tests.
>
> * New armv7 job is added (to replace armv6 one):
>   * FreeBSD-head-armv7-testvm
>   The images are available at https://artifact.ci.freebsd.org
>   FreeBSD-head-armv7-test is ready but needs test env update.
>
> ## Failing jobs
>
> * https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc6_build/
>   * See console log for the error details.
>
> ## Failing tests
>
> * https://ci.freebsd.org/job/FreeBSD-head-amd64-test/
>   * local.kyua.integration.cmd_about_test.topic__authors__installed
>   * sys.netipsec.tunnel.empty.v4
>   * sys.netipsec.tunnel.empty.v6
>   * sys.netpfil.common.forward.ipf_v4
>   * sys.netpfil.common.forward.ipfw_v4
>   * sys.netpfil.common.forward.pf_v4
>   * sys.netpfil.common.tos.ipfw_tos
>   * sys.netpfil.common.tos.pf_tos
>   * sys.netpfil.pf.forward.v4
I can’t actually reproduce this failure in my test VM, but with the ci 
test VM I can reproduce the problem.
It’s not related to pf, because the sanity check ping we do before we 
set up pf already fails.
Or rather pft_ping.py sends an incorrect packet, because `ping` does get 
the packet to go where it’s supposed to go.

Scapy seems to fail to find the source IP address, so we get this:

	12:12:22.152652 IP 0.0.0.0 > 198.51.100.3: ICMP echo request, id 0, seq 
0, length 12

I have a vague recollection that we’ve seem this problem before, but I 
can’t remember what we did about it.

In all likelihood most of the other netpfil tests fail for exactly the 
same reason.

Best regards,
Kristof
From owner-freebsd-testing@freebsd.org  Wed Apr 15 14:09:57 2020
Return-Path: <owner-freebsd-testing@freebsd.org>
Delivered-To: freebsd-testing@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id BFF8B2B7FD3;
 Wed, 15 Apr 2020 14:09:57 +0000 (UTC) (envelope-from kp@FreeBSD.org)
Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (4096 bits) client-digest SHA256)
 (Client CN "smtp.freebsd.org",
 Issuer "Let's Encrypt Authority X3" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 492PPd53FVz3PW8;
 Wed, 15 Apr 2020 14:09:57 +0000 (UTC) (envelope-from kp@FreeBSD.org)
Received: from venus.codepro.be (venus.codepro.be [5.9.86.228])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client CN "mx1.codepro.be", Issuer "Let's Encrypt Authority X3" (verified OK))
 (Authenticated sender: kp)
 by smtp.freebsd.org (Postfix) with ESMTPSA id 7A9EE14F66;
 Wed, 15 Apr 2020 14:09:57 +0000 (UTC) (envelope-from kp@FreeBSD.org)
Received: by venus.codepro.be (Postfix, authenticated sender kp) id 20857EAAA;
 Wed, 15 Apr 2020 16:09:56 +0200 (CEST)
From: "Kristof Provost" <kp@FreeBSD.org>
To: freebsd-testing@freebsd.org
Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org, bofh@freebsd.org, 
 "Alexander V. Chernikov" <melifaro@freebsd.org>
Subject: Re: FreeBSD CI Weekly Report 2020-04-12
Date: Wed, 15 Apr 2020 16:09:55 +0200
X-Mailer: MailMate (1.13.1r5671)
Message-ID: <DCC86D9B-ECF3-4393-B1C6-D76D1AE8BAC2@FreeBSD.org>
In-Reply-To: <A2DF53A3-FD86-427D-B1EE-508228B0F4CE@FreeBSD.org>
References: <20200414223710.GB33328@freefall.freebsd.org>
 <A2DF53A3-FD86-427D-B1EE-508228B0F4CE@FreeBSD.org>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: freebsd-testing@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Testing on FreeBSD <freebsd-testing.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-testing>, 
 <mailto:freebsd-testing-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-testing/>;
List-Post: <mailto:freebsd-testing@freebsd.org>
List-Help: <mailto:freebsd-testing-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-testing>, 
 <mailto:freebsd-testing-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 15 Apr 2020 14:09:57 -0000

On 15 Apr 2020, at 15:34, Kristof Provost wrote:
> On 15 Apr 2020, at 0:37, Li-Wen Hsu wrote:
>> (Please send the followup to freebsd-testing@ and note Reply-To is 
>> set.)
>>
>> FreeBSD CI Weekly Report 2020-04-12
>> ===================================
>>
>> Here is a summary of the FreeBSD Continuous Integration results for 
>> the period
>> from 2020-04-06 to 2020-04-12.
>>
>> During this period, we have:
>>
>> * 1801 builds (94.0% (+0.4) passed, 6.0% (-0.4) failed) of buildworld 
>> and
>>   buildkernel (GENERIC and LINT) were executed on aarch64, amd64, 
>> armv6,
>>   armv7, i386, mips, mips64, powerpc, powerpc64, powerpcspe, riscv64,
>>   sparc64 architectures for head, stable/12, stable/11 branches.
>> * 288 test runs (25.1% (-24.6) passed, 29.9% (+10.6) unstable, 45.1% 
>> (+14.1)
>>   exception) were executed on amd64, i386, riscv64 architectures for 
>> head,
>>   stable/12, stable/11 branches.
>> * 30 doc and www builds (83.3% (-1.3) passed, 16.7% (+1.3) failed)
>>
>> Test case status (on 2020-04-12 23:59):
>> | Branch/Architecture | Total     | Pass       | Fail     | Skipped  
>> |
>> | ------------------- | --------- | ---------- | -------- | -------- 
>> |
>> | head/amd64          | 7744 (+4) | 7638 (+19) | 14 (+5)  | 92 (-20) 
>> |
>> | head/i386           | 7742 (+4) | 7628 (+15) | 16 (+5)  | 98 (-16) 
>> |
>> | 12-STABLE/amd64     | 7508 (0)  | 7449 (-3)  | 1 (+1)   | 58 (+2)  
>> |
>> | 12-STABLE/i386      | 7506 (0)  | 7425 (-17) | 2 (+2)   | 79 (+15) 
>> |
>> | 11-STABLE/amd64     | 6882 (0)  | 6829 (-6)  | 1 (+1)   | 52 (+5)  
>> |
>> | 11-STABLE/i386      | 6880 (0)  | 6749 (-82) | 80 (+80) | 51 (+2)  
>> |
>>
>> (The statistics from experimental jobs are omitted)
>>
>> If any of the issues found by CI are in your area of interest or 
>> expertise
>> please investigate the PRs listed below.
>>
>> The latest web version of this report is available at
>> https://hackmd.io/@FreeBSD-CI/report-20200412 and archive is 
>> available at
>> https://hackmd.io/@FreeBSD-CI/ , any help is welcome.
>>
>> ## News
>>
>> * The test env now loads the required module for firewall tests.
>>
>> * New armv7 job is added (to replace armv6 one):
>>   * FreeBSD-head-armv7-testvm
>>   The images are available at https://artifact.ci.freebsd.org
>>   FreeBSD-head-armv7-test is ready but needs test env update.
>>
>> ## Failing jobs
>>
>> * https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc6_build/
>>   * See console log for the error details.
>>
>> ## Failing tests
>>
>> * https://ci.freebsd.org/job/FreeBSD-head-amd64-test/
>>   * local.kyua.integration.cmd_about_test.topic__authors__installed
>>   * sys.netipsec.tunnel.empty.v4
>>   * sys.netipsec.tunnel.empty.v6
>>   * sys.netpfil.common.forward.ipf_v4
>>   * sys.netpfil.common.forward.ipfw_v4
>>   * sys.netpfil.common.forward.pf_v4
>>   * sys.netpfil.common.tos.ipfw_tos
>>   * sys.netpfil.common.tos.pf_tos
>>   * sys.netpfil.pf.forward.v4
> I can’t actually reproduce this failure in my test VM, but with the 
> ci test VM I can reproduce the problem.
> It’s not related to pf, because the sanity check ping we do before 
> we set up pf already fails.
> Or rather pft_ping.py sends an incorrect packet, because `ping` does 
> get the packet to go where it’s supposed to go.
>
> Scapy seems to fail to find the source IP address, so we get this:
>
> 	12:12:22.152652 IP 0.0.0.0 > 198.51.100.3: ICMP echo request, id 0, 
> seq 0, length 12
>
> I have a vague recollection that we’ve seem this problem before, but 
> I can’t remember what we did about it.
>
> In all likelihood most of the other netpfil tests fail for exactly the 
> same reason.

The problem appears to be that 
/usr/local/lib/python3.7/site-packages/scapy/arch/unix.py is misparsing 
the `netstat -rnW` output.

For reference, this is the output in the test VM:

	Routing tables

	Internet:
	Destination        Gateway            Flags   Nhop#    Mtu      Netif 
Expire
	127.0.0.1          link#2             UH          1  16384        lo0
	192.0.2.0/24       link#4             U           2   1500    epair0a
	192.0.2.1          link#4             UHS         1  16384        lo0
	198.51.100.0/24    192.0.2.2          UGS         3   1500    epair0a

	Internet6:
	Destination                       Gateway                       Flags   
Nhop#    Mtu    Netif Expire
	::/96                             ::1                           UGRS    
     4  16384      lo0
	::1                               link#2                        UH      
     1  16384      lo0
	::ffff:0.0.0.0/96                 ::1                           UGRS    
     4  16384      lo0
	fe80::/10                         ::1                           UGRS    
     4  16384      lo0
	fe80::%lo0/64                     link#2                        U       
     3  16384      lo0
	fe80::1%lo0                       link#2                        UHS     
     2  16384      lo0
	fe80::%epair0a/64                 link#4                        U       
     5   1500  epair0a
	fe80::3d:9dff:fe7c:d70a%epair0a   link#4                        UHS     
     1  16384      lo0
	fe80::%epair1a/64                 link#6                        U       
     6   1500  epair1a
	fe80::37:9eff:fe03:250a%epair1a   link#6                        UHS     
     1  16384      lo0
	ff02::/16                         ::1                           UGRS    
     4  16384      lo0

The parsing code seems to think that the netif for the 198.51.100.0/24 
route is 1500 rather than epair0a.
This may be related to the difference in netstat output, because on my 
VM it looks like this:

	Routing tables

	Internet:
	Destination        Gateway            Flags       Use    Mtu      Netif 
Expire
	default            172.16.2.1         UGS         319   1500     vtnet0
	127.0.0.1          link#2             UH            0  16384        lo0
	172.16.2.0/24      link#1             U            14   1500     vtnet0
	172.16.2.2         link#1             UHS           0  16384        lo0

	Internet6:
	Destination                       Gateway                       Flags   
     Use    Mtu    Netif Expire
	::/96                             ::1                           UGRS    
       0  16384      lo0
	::1                               link#2                        UH      
       0  16384      lo0
	::ffff:0.0.0.0/96                 ::1                           UGRS    
       0  16384      lo0
	fe80::/10                         ::1                           UGRS    
       0  16384      lo0
	fe80::%vtnet0/64                  link#1                        U       
       0   1500   vtnet0
	fe80::5a9c:fcff:fe02:a95e%vtnet0  link#1                        UHS     
       0  16384      lo0
	fe80::%lo0/64                     link#2                        U       
       0  16384      lo0
	fe80::1%lo0                       link#2                        UHS     
       0  16384      lo0
	ff02::/16                         ::1                           UGRS    
       0  16384      lo0

I suspect that this change was introduced in r359823 (Introduce nexthop 
objects and new routing KPI).

Best regards,
Kristof
From owner-freebsd-testing@freebsd.org  Wed Apr 15 14:49:44 2020
Return-Path: <owner-freebsd-testing@freebsd.org>
Delivered-To: freebsd-testing@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4048E2B8F3B;
 Wed, 15 Apr 2020 14:49:44 +0000 (UTC)
 (envelope-from olivier@freebsd.org)
Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (4096 bits) client-digest SHA256)
 (Client CN "smtp.freebsd.org",
 Issuer "Let's Encrypt Authority X3" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 492QHX0xqyz3wt2;
 Wed, 15 Apr 2020 14:49:44 +0000 (UTC)
 (envelope-from olivier@freebsd.org)
Received: from mail-pg1-f179.google.com (mail-pg1-f179.google.com
 [209.85.215.179])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 (Authenticated sender: olivier/mail)
 by smtp.freebsd.org (Postfix) with ESMTPSA id E6B86170E7;
 Wed, 15 Apr 2020 14:49:43 +0000 (UTC)
 (envelope-from olivier@freebsd.org)
Received: by mail-pg1-f179.google.com with SMTP id r4so17093pgg.4;
 Wed, 15 Apr 2020 07:49:43 -0700 (PDT)
X-Gm-Message-State: AGi0PuYCsN/5awg9udJTfwL7GvOf1DjTBmnISu7CpUvEVvY6WRFj07ag
 imRgxkdYAlQg+6ygtL28GIdD2tOMvK/So9jB1CM=
X-Google-Smtp-Source: APiQypKecHW77MF5DIXi8FXRktWsuQds0xTOB4APspMaWMrkHF4euHjE8RZvfkDfLkoNh/Z3pJhQk8izB38QA7UpQLQ=
X-Received: by 2002:a62:4e0c:: with SMTP id c12mr27056707pfb.87.1586962182720; 
 Wed, 15 Apr 2020 07:49:42 -0700 (PDT)
MIME-Version: 1.0
References: <20200414223710.GB33328@freefall.freebsd.org>
 <A2DF53A3-FD86-427D-B1EE-508228B0F4CE@FreeBSD.org>
 <DCC86D9B-ECF3-4393-B1C6-D76D1AE8BAC2@FreeBSD.org>
In-Reply-To: <DCC86D9B-ECF3-4393-B1C6-D76D1AE8BAC2@FreeBSD.org>
From: =?UTF-8?Q?Olivier_Cochard=2DLabb=C3=A9?= <olivier@freebsd.org>
Date: Wed, 15 Apr 2020 16:49:31 +0200
X-Gmail-Original-Message-ID: <CA+q+Tcr-B5dzPKV-DSiDYmcueXX=AFgh6+j0G=-xJYB0XBzBpQ@mail.gmail.com>
Message-ID: <CA+q+Tcr-B5dzPKV-DSiDYmcueXX=AFgh6+j0G=-xJYB0XBzBpQ@mail.gmail.com>
Subject: Re: FreeBSD CI Weekly Report 2020-04-12
To: Kristof Provost <kp@freebsd.org>
Cc: freebsd-testing@freebsd.org, freebsd-current <freebsd-current@freebsd.org>,
 "freebsd-stable@freebsd.org Stable" <freebsd-stable@freebsd.org>,
 bofh@freebsd.org, "Alexander V. Chernikov" <melifaro@freebsd.org>
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: freebsd-testing@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Testing on FreeBSD <freebsd-testing.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-testing>, 
 <mailto:freebsd-testing-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-testing/>;
List-Post: <mailto:freebsd-testing@freebsd.org>
List-Help: <mailto:freebsd-testing-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-testing>, 
 <mailto:freebsd-testing-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 15 Apr 2020 14:49:44 -0000

On Wed, Apr 15, 2020 at 4:10 PM Kristof Provost <kp@freebsd.org> wrote:

>
> The problem appears to be that
> /usr/local/lib/python3.7/site-packages/scapy/arch/unix.py is misparsing
> the `netstat -rnW` output.
>

Shouldn't scapy use the libxo output of netstat to mitigate this regression
?



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?A2DF53A3-FD86-427D-B1EE-508228B0F4CE>