Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 5 Mar 2017 18:03:46 -0800
From:      "Tony Hain" <tony@tndh.net>
To:        <freebsd-arm@freebsd.org>
Cc:        "'Ian Lepore'" <ian@freebsd.org>
Subject:   BBB asymmetric stack?
Message-ID:  <01c701d2961d$e3dd6100$ab982300$@tndh.net>

next in thread | raw e-mail | index | archive | help
FreeBSD 12.0-CURRENT #0 r314118M: Wed Feb 22 22:49:19 PST 2017  

Trying to calibrate the ntp feed on the BBB I was suspecting some problem
with the dmtpps driver, but then realized that network asymmetry would be
more likely to explain the persistent offset. It would appear that there is
an asymmetry within the stack on the am335x based system.

ping6 Rpi -> BBB
10 packets transmitted, 10 received, 0% packet loss, time 9006ms
rtt min/avg/max/mdev = 0.503/0.567/0.642/0.047 ms

ping6 BBB -> Rpi
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.878/0.967/1.141/0.134 ms

Those are on adjacent ports on the same 100mbps switch. 

---

ping6 Fbsd-8 -> BBB
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.599/0.734/1.041/0.112 ms

ping6 BBB -> Fbsd-8
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.855/0.970/1.816/0.296 ms

Those are one router hop apart.

---

Even localhost appears to indicate a problem in the FreeBSD-arm side of
this:
pi# ping6 ::1
PING ::1(::1) 56 data bytes
64 bytes from ::1: icmp_seq=1 ttl=64 time=0.220 ms
64 bytes from ::1: icmp_seq=2 ttl=64 time=0.222 ms
64 bytes from ::1: icmp_seq=3 ttl=64 time=0.221 ms

i386# ping6 ::1
PING6(56=40+8+8 bytes) ::1 --> ::1
16 bytes from ::1, icmp_seq=0 hlim=64 time=0.185 ms
16 bytes from ::1, icmp_seq=1 hlim=64 time=0.135 ms
16 bytes from ::1, icmp_seq=2 hlim=64 time=0.129 ms

bbb# ping6 ::1
PING6(56=40+8+8 bytes) ::1 --> ::1
16 bytes from ::1, icmp_seq=0 hlim=64 time=0.672 ms
16 bytes from ::1, icmp_seq=1 hlim=64 time=0.477 ms
16 bytes from ::1, icmp_seq=2 hlim=64 time=0.464 ms


I realize the loopback ping by itself is not a good indicator of an
asymmetry problem, but FreeBSD 8 on i386 systems ping::1 at ~ 130us, while
all the FreeBSD 10/11 VMs (Hyper-V & VMware) and the pi ping ::1 at ~220us.
The issue is that the time is always longer when the BBB initiates the ping,
which would imply some transmit delay that does not exist when responding to
an incoming ping, but it could be simply getting the reply back to user
space which only happens in the BBB initiate case. 

====
The ntp perspective might shed some light on which direction.

pi
pi# ntpq -p
     remote           refid      st t when poll reach   delay   offset
jitter
============================================================================
==
 GPS_NMEA(0)     .GPS.            0 s 342d   16    0    0.000  -29.245
0.000
oPPS(0)          .PPS.            0 s    6   16  377    0.000    0.000
0.002
*172.20.0.18     .PPS.            1 u   12   64  377    0.492    0.030
0.026                  <== i386

i386
# ntpq -p
     remote           refid      st t when poll reach   delay   offset
jitter
============================================================================
==
xPPS(1)          .PPS.            0 l   14   16  377    0.000    0.001
0.002
oGPS_NMEA(1)     .GPS.            0 l    8   16  377    0.000    0.000
0.002
+2001:470:e930:2  .PPS.            1 u   30   64  377    0.519    0.008
0.020          <==pi

BBB
# ntpq -p
     remote           refid      st t when poll reach   delay   offset
jitter
============================================================================
==
oPPS(0)          .PPS.            0 s   32   32  377    0.000    0.000
0.004
*GPS_NMEA(0)     .GPS.            0 s    7    8  377    0.000  -47.650
39.504
+2001:470:e930:2 .PPS.            1 u   28   32  377    0.682   -0.164
0.043         <== pi
+2001:470:e930:7 .GPS.            1 u   44   64  377    0.963   -0.197
1.771        <== i386

The offset being about half of the path variance for an asymmetric path is
expected ntp behavior. All of these systems are looking at the same PPS
pulse, though the i386 is seeing it through an RS232 driver delay so it has
an additional 1.3us fudge factor.  

In any case, it appears that there is some form of delay in getting bits out
of the box. 

Tony





Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?01c701d2961d$e3dd6100$ab982300$>