From owner-freebsd-current@freebsd.org Fri Aug 28 12:04:25 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 279013D769D for ; Fri, 28 Aug 2020 12:04:25 +0000 (UTC) (envelope-from strigub@dialektika.com) Received: from strigub.esotery.net (strigub.esotery.net [109.251.97.75]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BdJDS0LQdz4Srn for ; Fri, 28 Aug 2020 12:04:23 +0000 (UTC) (envelope-from strigub@dialektika.com) Received: from [10.10.10.163] (helo=SERJHOME) by strigub.esotery.net with smtp (Exim 4.94 (FreeBSD)) (envelope-from ) id 1kBd6z-0001rr-Be for freebsd-current@freebsd.org; Fri, 28 Aug 2020 15:04:15 +0300 Message-ID: <4C267E15E99E4003964861B4B7ADC9E3@SERJHOME> Reply-To: "S.N. Trigub" From: "S.N. Trigub" To: Date: Fri, 28 Aug 2020 15:02:46 +0300 Organization: =?utf-8?B?0JfQkNCeICLQlNC40LDQu9C10LrRgtC40LrQsCI=?= MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="utf-8"; reply-type=original Content-Transfer-Encoding: 8bit X-Priority: 3 X-MSMail-Priority: Normal Importance: Normal X-Mailer: Microsoft Windows Live Mail 15.4.3555.308 X-MimeOLE: Produced By Microsoft MimeOLE V15.4.3555.308 X-SA-Exim-Connect-IP: 10.10.10.163 X-SA-Exim-Mail-From: strigub@dialektika.com X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on strigub.esotery.net X-Spam-Level: * X-Spam-Status: No, score=1.1 required=5.0 tests=ALL_TRUSTED,STOX_REPLY_TYPE, STOX_REPLY_TYPE_WITHOUT_QUOTES autolearn=no autolearn_force=no version=3.4.4 Subject: speedtest.net in multi connections mode causes the FreeBSD 13-CURRENT router to crash X-SA-Exim-Version: 4.2.1 X-SA-Exim-Scanned: Yes (on strigub.esotery.net) X-Rspamd-Queue-Id: 4BdJDS0LQdz4Srn X-Spamd-Bar: + X-Spamd-Result: default: False [1.37 / 15.00]; HAS_REPLYTO(0.00)[strigub@dialektika.com]; ARC_NA(0.00)[]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; STOX_REPLY_TYPE(1.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[dialektika.com]; RCPT_COUNT_ONE(0.00)[1]; HAS_ORG_HEADER(0.00)[]; RCVD_TLS_LAST(0.00)[]; NEURAL_SPAM_SHORT(0.11)[0.111]; NEURAL_SPAM_LONG(0.10)[0.101]; HAS_X_PRIO_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-0.04)[-0.042]; MID_RHS_NOT_FQDN(0.50)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:31148, ipnet:109.251.97.0/24, country:UA]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Aug 2020 12:04:25 -0000 Hi! I run FreeBSD 13.0-CURRENT #0 r363772 as router in small office with two network interface: sk0 and sk1. sk0 connected to 2 providers (main and backup) via VLAN and natd, sk1 connected to office network via VLAN. I had to use VLAN due to cabling issues. As soon as the user runs the speedtest.net in browser on any client computer in network FreeBSD router goes into panic mode and automatically reboot during multi connection upload test. In single connections mode of speedtest.net all is fine. Error is : Fatal trap 12: page fault while in kernel mode. In my opinion the error is caused by multiple openings of outgoing connections on router in a short period of time. This is not a hardware error. I tested it on three different motherboards with different kernel versions and diffenent amount of RAM (8GB DDR2 and 16, 24, 32GB DDR3). Maybe this is somehow due to the use of VLAN, or network card driver problem... I noticed this problem about six months ago when I switched from FreeBSD 12-Current to FreeBSD 13-Current. Since then I have been updating the kernel about once a week, but the error still present. I read on the some artictes in Internet that this can happen when tcp extensions are enabled. I disabled tcp_extensons in rc.conf, and set some sysctls params as described in Internet-articles, but it didn't affect anything. The same error sometimes occurs when clients are talking on Skype. Could you help me? Any idea is appreciated! Thank you. Sergei. ====================My network cards:================================================================ skc0: port 0xc100-0xc1ff mem 0xf7b44000-0xf7b47fff irq 16 at device 1.0 on pci6 skc0: DGE-530T Gigabit Ethernet Adapter rev. (0x9) sk0: on skc0 sk0: Ethernet address: 00:15:e9:b8:c3:53 miibus1: on sk0 e1000phy0: PHY 0 on miibus1 e1000phy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto skc1: port 0xc000-0xc0ff mem 0xf7b40000-0xf7b43fff irq 17 at device 2.0 on pci6 skc1: DGE-530T Gigabit Ethernet Adapter rev. (0x9) sk1: on skc1 sk1: Ethernet address: 00:19:5b:8a:97:a7 miibus2: on sk1 e1000phy1: PHY 0 on miibus2 e1000phy1: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto ================================ My natd.conf:==================================================================== instance default interface vlan2 port 8668 use_sockets yes same_ports yes # instance vlan3 interface vlan3 port 8669 use_sockets yes same_ports yes # globalport 8670 # # ===================== My rc.conf:========================================================================= defaultrouter="109.251.97.65" tcp_extensions="NO" # Set to NO to turn off RFC1323 extensions. # ifconfig_sk0="up -rxcsum -txcsum" ifconfig_sk1="up -rxcsum -txcsum" ipv6_network_interfaces="none" # Default is auto ipv6_activate_all_interfaces="NO" # this is the default ip6addrctl_enable="NO" # New way to disable IPv6 support ip6addrctl_policy="ipv4_prefer" # Use IPv4 instead of IPv6 cloned_interfaces="vlan2 vlan3 vlan5" # # vlan2 – provider 1 as main # vlan3 -- provider 2 as backup # vlan5 – office network # ifconfig_vlan2="inet 109.251.97.75 netmask 255.255.255.224 vlan 2 vlandev sk0" ifconfig_vlan3="inet 95.164.8.37 netmask 255.255.252.0 vlan 3 vlandev sk0" ifconfig_vlan5="inet 10.10.10.177 netmask 255.255.255.240 vlan 5 vlandev sk1" # gateway_enable="YES" # zfs_enable="YES" # Set to YES to automatically mount ZFS file systems natd_enable="YES" # enables NAT natd_flags="-f /etc/natd.conf" ============== My sysctl.conf:=============================================================================== debug.debugger_on_panic=0 kern.coredump=0 kern.corefile=/dev/null net.inet.ip.fw.one_pass=1 net.inet.tcp.blackhole=2 net.inet.udp.blackhole=1 net.inet.tcp.recvspace=1048576 net.inet.tcp.sendspace=524288 net.inet.tcp.recvbuf_max=4194304 net.inet.tcp.sendbuf_inc=65535 net.inet.tcp.msl=5000 net.inet.ip.fw.dyn_buckets=65536 net.inet.ip.fw.dyn_max=65536 net.inet.ip.fw.dyn_ack_lifetime=120 net.inet.ip.fw.dyn_syn_lifetime=10 net.inet.ip.fw.dyn_fin_lifetime=2 net.inet.ip.fw.dyn_short_lifetime=10 ========================= My loader.conf:============================================================== zfs_load="YES" vfs.root.mountfrom="zfs:zpoolasr" vfs.zfs.prefetch_disable=0 accf_data_load="YES" # Wait for data accept filter accf_dns_load="YES" # Wait for full DNS request accept filter accf_http_load="YES" # Wait for full HTTP request accept filter coretemp_load="YES" hw.vga.textmode=0 ============================= My kernel config:========================================================== include GENERIC # ident HOME64 # nooptions KDB # Enable kernel debugger support. nooptions KDB_TRACE # Print a stack trace for a panic. nooptions DDB # Support DDB. nooptions GDB # Support remote GDB. options KDB_UNATTENDED # Don't enter debugger after panic nooptions INVARIANTS # Enable calls of extra sanity checking nooptions INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS nooptions WITNESS # Enable checks to detect deadlocks and cycles nooptions WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed nooptions WITNESS_KDB nooptions SOCKBUF_DEBUG nooptions DIAGNOSTIC nooptions DEBUG_MEMGUARD nooptions DEBUG_VFS_LOCKS # device smb device smbus # System management bus device intpm # Intel power management device iicbus # I2C bus system device iicsmb # I2C to SMB bridge device iicbb # I2C generic bit-banging drive device ichsmb # Intel ICH SMBus controller chips (82801AA, 82801AB, 82801BA) # options ALTQ options ALTQ_CBQ options ALTQ_RED options ALTQ_RIO options ALTQ_HFSC options ALTQ_PRIQ options ALTQ_NOPCC options MROUTING # Multicast routing options ROUTETABLES=4 # max 16. 1 is back compatible. options IPFILTER # # FIREWALL # options IPFIREWALL options IPFIREWALL_DEFAULT_TO_ACCEPT options IPFIREWALL_NAT #ipfw kernel nat support options IPSTEALTH #support for stealth forwarding options TCPDEBUG options IPDIVERT #divert sockets options TCP_SIGNATURE #include support for RFC 2385 # # NETGRAPH # options NETGRAPH options NETGRAPH_ETHER options NETGRAPH_SOCKET options NETGRAPH_MPPC_ENCRYPTION options NETGRAPH_MPPC_COMPRESSION options NETGRAPH_BPF options NETGRAPH_IFACE options NETGRAPH_KSOCKET options NETGRAPH_PPP options NETGRAPH_PPTPGRE options NETGRAPH_TCPMSS options NETGRAPH_VJC options NETGRAPH_ONE2MANY options NETGRAPH_RFC1490 options NETGRAPH_TEE options NETGRAPH_TTY options NETGRAPH_UI # libalias library, performing NAT options LIBALIAS options LIBICONV # mchain library. It can be either loaded as KLD or compiled into kernel options LIBMCHAIN #options FLOWTABLE # per-cpu routing cache options DUMMYNET options HZ=1000 # Cryptographically secure random number generator; /dev/random device cryptodev # /dev/crypto for access to h/w device rndtest # FIPS 140-2 entropy tester options GEOM_BDE # Disk encryption. options GEOM_ELI # Disk encryption. ======================= My dmesg.boot:==================================================================== VERBOSE_SYSINIT: DDB not enabled, symbol lookups disabled. Copyright (c) 1992-2020 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 13.0-CURRENT #0 r363772: Sun Aug 2 20:37:24 EEST 2020 root@strigub.esotery.net:/usr/obj/usr/src/amd64.amd64/sys/HOME64 amd64 FreeBSD clang version 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-rc2-0-g77d76b71d7d) VT(vga): resolution 640x480 VERBOSE_SYSINIT: DDB not enabled, symbol lookups disabled. CPU: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (3411.20-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306a9 Family=0x6 Model=0x3a Stepping=9 Features=0xbfebfbff Features2=0x7fbae3ff AMD Features=0x28100800 AMD Features2=0x1 Structured Extended Features=0x281 XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 33295888384 (31753 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 hardware threads random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" random: unblocking device. ioapic0 irqs 0-23 Launching APs: 1 7 6 2 4 5 3 Timecounter "TSC-low" frequency 1705600786 Hz quality 1000 random: entropy device external interface 000.000018 [4342] netmap_init netmap: loaded module [ath_hal] loaded WARNING: Device "kbd" is Giant locked and may be deleted before FreeBSD 13.0. kbd1 at kbdmux0 nexus0 vtvga0: cryptosoft0: acpi0: acpi0: Power Button (fixed) cpu0: on acpi0 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 550 atrtc0: port 0x70-0x77 irq 8 on acpi0 atrtc0: Warning: Couldn't map I/O. atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 acpi_ec0: port 0x62,0x66 on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 16 at device 1.0 on pci0 pci1: on pcib1 vgapci0: port 0xe000-0xe0ff mem 0xe0000000-0xefffffff,0xf7d20000-0xf7d2ffff irq 16 at device 0.0 on pci1 vgapci0: Boot video device hdac0: mem 0xf7d30000-0xf7d33fff irq 17 at device 0.1 on pci1 pci0: at device 22.0 (no driver attached) ehci0: mem 0xf7e08000-0xf7e083ff irq 23 at device 26.0 on pci0 usbus0: EHCI version 1.0 usbus0 on ehci0 usbus0: 480Mbps High Speed USB v2.0 hdac1: mem 0xf7e00000-0xf7e03fff irq 22 at device 27.0 on pci0 pcib2: irq 16 at device 28.0 on pci0 pci2: on pcib2 aac0: mem 0xf7800000-0xf79fffff irq 16 at device 0.0 on pci2 aac0: Enabling 64-bit address support aac0: Enable Raw I/O aac0: Enable 64-bit array aac0: New comm. interface enabled aac0: Adaptec 5805Z, aac driver 2.1.9-1 aacp0 on aac0 aacp1 on aac0 aacp2 on aac0 pcib3: irq 17 at device 28.5 on pci0 pci3: on pcib3 xhci0: mem 0xf7c00000-0xf7c07fff irq 17 at device 0.0 on pci3 xhci0: 32 bytes context size, 32-bit DMA xhci0: Unable to map MSI-X table usbus1 on xhci0 usbus1: 5.0Gbps Super Speed USB v3.0 pcib4: irq 18 at device 28.6 on pci0 pci4: on pcib4 re0: port 0xd000-0xd0ff mem 0xf0004000-0xf0004fff,0xf0000000-0xf0003fff irq 18 at device 0.0 on pci4 re0: Using 1 MSI-X message re0: Chip rev. 0x2c800000 re0: MAC rev. 0x00100000 miibus0: on re0 rgephy0: PHY 1 on miibus0 rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow re0: Using defaults for TSO: 65518/35/2048 re0: Ethernet address: 00:0b:0e:0f:00:ed re0: netmap queues/slots: TX 1/256, RX 1/256 pcib5: irq 19 at device 28.7 on pci0 pci5: on pcib5 pcib6: irq 19 at device 0.0 on pci5 pci6: on pcib6 skc0: port 0xc100-0xc1ff mem 0xf7b44000-0xf7b47fff irq 16 at device 1.0 on pci6 skc0: DGE-530T Gigabit Ethernet Adapter rev. (0x9) sk0: on skc0 sk0: Ethernet address: 00:15:e9:b8:c3:53 miibus1: on sk0 e1000phy0: PHY 0 on miibus1 e1000phy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto skc1: port 0xc000-0xc0ff mem 0xf7b40000-0xf7b43fff irq 17 at device 2.0 on pci6 skc1: DGE-530T Gigabit Ethernet Adapter rev. (0x9) sk1: on skc1 sk1: Ethernet address: 00:19:5b:8a:97:a7 miibus2: on sk1 e1000phy1: PHY 0 on miibus2 e1000phy1: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto ehci1: mem 0xf7e07000-0xf7e073ff irq 23 at device 29.0 on pci0 usbus2: EHCI version 1.0 usbus2 on ehci1 usbus2: 480Mbps High Speed USB v2.0 isab0: at device 31.0 on pci0 isa0: on isab0 ahci0: port 0xf070-0xf077,0xf060-0xf063,0xf050-0xf057,0xf040-0xf043,0xf020-0xf03f mem 0xf7e06000-0xf7e067ff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported ahcich5: at channel 5 on ahci0 ahciem0: on ahci0 ichsmb0: port 0xf000-0xf01f mem 0xf7e05000-0xf7e050ff irq 18 at device 31.3 on pci0 acpi_button0: on acpi0 acpi_tz0: on acpi0 acpi_tz1: on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 orm0: at iomem 0xc0000-0xcffff,0xd0000-0xd67ff pnpid ORM0000 on isa0 coretemp0: on cpu0 est0: on cpu0 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec IP Filter: v5.1.2 initialized. Default = pass all, Logging = disabled ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled load_dn_sched dn_sched FQ_PIE loaded load_dn_sched dn_sched PRIO loaded load_dn_sched dn_sched QFQ loaded load_dn_sched dn_sched RR loaded load_dn_sched dn_sched WF2Q+ loaded load_dn_sched dn_sched FIFO loaded load_dn_sched dn_sched FQ_CODEL loaded load_dn_aqm dn_aqm CODEL loaded load_dn_aqm dn_aqm PIE loaded Obsolete code will be removed soon: random(9) is the obsolete Park-Miller LCG from 1988 hdacc0: at cad 0 on hdac0 hdaa0: at nid 1 on hdacc0 pcm0: at nid 3 on hdaa0 hdacc1: at cad 0 on hdac1 hdaa1: at nid 1 on hdacc1 pcm1: at nid 20,22,21,23 and 24,26 on hdaa1 pcm2: at nid 27 and 25 on hdaa1 pcm3: at nid 30 on hdaa1 pcm4: at nid 17 on hdaa1 aacd0 on aac0 aacd0: 11443190MB (23435653120 sectors) ugen0.1: at usbus0 smbus0: on ichsmb0 smb0: on smbus0 Trying to mount root from zfs:zpoolasr []... ugen2.1: at usbus2 uhub0 on usbus2 uhub0: on usbus2 uhub1 on usbus0 uhub1: on usbus0 ugen1.1: <0x1b21 XHCI root HUB> at usbus1 uhub2 on usbus1 uhub2: <0x1b21 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus1 ses0 at ahciem0 bus 0 scbus4 target 0 lun 0 ses0: SEMB S-E-S 2.00 device ses0: SEMB SES Device ada0 at ahcich5 bus 0 scbus3 target 0 lun 0 ada0: ATA8-ACS SATA 3.x device ada0: Serial Number JG40006EG6NTMC ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 953869MB (1953525168 512 byte sectors) ses0: ada0 in 'Slot 05', SATA Slot: scbus3 target 0 pass0 at aacp0 bus 0 scbus0 target 0 lun 0 pass0: Fixed Uninstalled SPC-3 SCSI device pass0: Serial Number VBG2MV2R pass0: 3.300MB/s transfers pass1 at aacp0 bus 0 scbus0 target 1 lun 0 pass1: Fixed Uninstalled SPC-3 SCSI device pass1: Serial Number VBG2G5ZR pass1: 3.300MB/s transfers pass2 at aacp0 bus 0 scbus0 target 2 lun 0 pass2: Fixed Uninstalled SPC-3 SCSI device pass2: Serial Number VBG2G2HR pass2: 3.300MB/s transfers pass3 at aacp0 bus 0 scbus0 target 3 lun 0 pass3: Fixed Uninstalled SPC-3 SCSI device pass3: Serial Number VBG2HMVR pass3: 3.300MB/s transfers Root mount waiting for: usbus0 usbus1 usbus2 uhub2: 4 ports with 4 removable, self powered uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered Root mount waiting for: usbus0 usbus2 ugen0.2: at usbus0 uhub3 on uhub1 uhub3: on usbus0 Root mount waiting for: usbus0 usbus2 ugen2.2: at usbus2 uhub4 on uhub0 uhub4: on usbus2 uhub3: 6 ports with 6 removable, self powered Root mount waiting for: usbus2 uhub4: 8 ports with 8 removable, self powered