Date: Thu, 29 Feb 2024 09:24:14 +0000 From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 277389] Reproduceable low memory freeze on 14.0-RELEASE-p5 Message-ID: <bug-277389-227@https.bugs.freebsd.org/bugzilla/>
next in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D277389 Bug ID: 277389 Summary: Reproduceable low memory freeze on 14.0-RELEASE-p5 Product: Base System Version: 14.0-RELEASE Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: pascal.guitierrez@gmail.com I can reliably reproduce a freeze using iozone on 14.0-RELEASE-p5 using a simple ZFS pool backed onto SATA HDDs as such: NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP=20= =20=20 HEALTH ALTROOT tank 1.81T 512G 1.31T - - 36% 27% 1.00x=20= =20=20 ONLINE - mirror-0 1.81T 512G 1.31T - - 36% 27.6% -=20= =20=20 ONLINE ada0 1.82T - - - - - - -=20= =20=20 ONLINE ada1 1.82T - - - - - - -=20= =20=20 ONLINE To reproduce I create a ZFS filesystem with compression=3Doff, then run ioz= one -i 0,1 -l 512 -r 4k -s 1g The write tests execute fine (writing out 512 1g files), however when it ru= ns the read tests (1=3Dread/re-read) the system freezes due to an apparent low memory condition after only a few minutes. I've tried increasing the following sysctls from their defaults: vm.v_free_target vm.v_free_min vm.v_free_severe vm.v_free_reserved vfs.zfs.arc.sys_free vfs.zfs.arc.free_target and reducing the following: vfs.zfs.arc_max But no effect, free memory reliably drops below 100M and does not recover a= nd soon after the system will freeze and require a hard reset. This machine has 16GB of RAM with no swap (have also tried with swap on zvol but same effect), dmesg is below: ---<<BOOT>>--- Copyright (c) 1992-2023 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 14.0-RELEASE-p5 #0: Tue Feb 13 23:37:36 UTC 2024 root@amd64-builder.daemonology.net:/usr/obj/usr/src/amd64.amd64/sys/GEN= ERIC amd64 FreeBSD clang version 16.0.6 (https://github.com/llvm/llvm-project.git llvmorg-16.0.6-0-g7cbf1a259152) VT(efifb): resolution 800x600 CPU: Intel(R) Atom(TM) CPU C3758 @ 2.20GHz (2200.21-MHz K8-class CPU) Origin=3D"GenuineIntel" Id=3D0x506f1 Family=3D0x6 Model=3D0x5f Steppi= ng=3D1 =20 Features=3D0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,= MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> =20 Features2=3D0x4ff8ebbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,S= DBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSX= SAVE,RDRAND> AMD Features=3D0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM> AMD Features2=3D0x101<LAHF,Prefetch> Structured Extended Features=3D0x2294e283<FSGSBASE,TSCADJ,SMEP,ERMS,NFPUSG,MPX,PQE,RDSEED,SMAP,= CLFLUSHOPT,PROCTRACE,SHA> Structured Extended Features3=3D0xac000400<MD_CLEAR,IBPB,STIBP,ARCH_CAP,S= SBD> XSAVE Features=3D0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES> IA32_ARCH_CAPS=3D0x14000c69<RDCL_NO,SKIP_L1DFL_VME,MDS_NO> VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory =3D 17179869184 (16384 MB) avail memory =3D 16552595456 (15785 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: <INTEL TIANO > WARNING: L1 data cache covers fewer APIC IDs than a core (0 < 1) FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 1 package(s) x 8 core(s) random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" random: unblocking device. ioapic0 <Version 2.0> irqs 0-23 Launching APs: 2 3 5 1 4 6 7 random: entropy device external interface kbd0 at kbdmux0 efirtc0: <EFI Realtime Clock> efirtc0: registered as a time-of-day clock, resolution 1.000000s smbios0: <System Management BIOS> at iomem 0x7f0c4000-0x7f0c401e smbios0: Version: 3.0, BCD Revision: 3.0 aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS,SHA1,SHA256> acpi0: <SUPERM SMCI--MB> acpi0: Power Button (fixed) cpu0: <ACPI CPU> on acpi0 hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 24000000 Hz quality 950 Event timer "HPET" frequency 24000000 Hz quality 550 atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0 atrtc0: Warning: Couldn't map I/O. atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 pcib1: <ACPI PCI-PCI bridge> at device 6.0 on pci0 pci1: <ACPI PCI bus> on pcib1 pci1: <processor> at device 0.0 (no driver attached) pcib2: <ACPI PCI-PCI bridge> mem 0xdf440000-0xdf45ffff at device 9.0 on pci0 pci2: <ACPI PCI bus> on pcib2 pcib3: <ACPI PCI-PCI bridge> mem 0xdf420000-0xdf43ffff at device 16.0 on pc= i0 pci3: <ACPI PCI bus> on pcib3 pcib4: <ACPI PCI-PCI bridge> mem 0xdf400000-0xdf41ffff at device 17.0 on pc= i0 pci4: <ACPI PCI bus> on pcib4 pcib5: <ACPI PCI-PCI bridge> at device 0.0 on pci4 pci5: <ACPI PCI bus> on pcib5 vgapci0: <VGA-compatible display> port 0xd000-0xd07f mem 0xde000000-0xdeffffff,0xdf000000-0xdf01ffff at device 0.0 on pci5 ahci0: <Intel Denverton AHCI SATA controller> port 0xe090-0xe097,0xe080-0xe083,0xe040-0xe05f mem 0xdf476000-0xdf477fff,0xdf47d000-0xdf47d0ff,0xdf47c000-0xdf47c7ff at device 19.0 on pci0 ahci0: AHCI v1.31 with 8 6Gbps ports, Port Multiplier not supported ahcich0: <AHCI channel> at channel 0 on ahci0 ahcich1: <AHCI channel> at channel 1 on ahci0 ahcich2: <AHCI channel> at channel 2 on ahci0 ahcich3: <AHCI channel> at channel 3 on ahci0 ahcich4: <AHCI channel> at channel 4 on ahci0 ahcich5: <AHCI channel> at channel 5 on ahci0 ahcich6: <AHCI channel> at channel 6 on ahci0 ahcich7: <AHCI channel> at channel 7 on ahci0 ahciem0: <AHCI enclosure management bridge> on ahci0 ahci1: <Intel Denverton AHCI SATA controller> port 0xe070-0xe077,0xe060-0xe063,0xe020-0xe03f mem 0xdf474000-0xdf475fff,0xdf47b000-0xdf47b0ff,0xdf47a000-0xdf47a7ff at device 20.0 on pci0 ahci1: AHCI v1.31 with 4 6Gbps ports, Port Multiplier not supported ahcich11: <AHCI channel> at channel 3 on ahci1 ahcich12: <AHCI channel> at channel 4 on ahci1 ahcich13: <AHCI channel> at channel 5 on ahci1 ahcich14: <AHCI channel> at channel 6 on ahci1 ahciem1: <AHCI enclosure management bridge> on ahci1 xhci0: <Intel Denverton USB 3.0 controller> mem 0xdf460000-0xdf46ffff at de= vice 21.0 on pci0 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pcib6: <ACPI PCI-PCI bridge> at device 22.0 on pci0 pci6: <ACPI PCI bus> on pcib6 ix0: <Intel(R) X553 (1GbE)> mem 0xddc00000-0xdddfffff,0xdde04000-0xdde07fff= at device 0.0 on pci6 ix0: Using 2048 TX descriptors and 2048 RX descriptors ix0: Using 8 RX queues 8 TX queues ix0: Using MSI-X interrupts with 9 vectors ix0: allocated for 8 queues ix0: allocated for 8 rx queues ix0: Ethernet address: 3c:ec:ef:e9:aa:30 ix0: eTrack 0x80000877 ix0: netmap queues/slots: TX 8/2048, RX 8/2048 ix1: <Intel(R) X553 (1GbE)> mem 0xdda00000-0xddbfffff,0xdde00000-0xdde03fff= at device 0.1 on pci6 ix1: Using 2048 TX descriptors and 2048 RX descriptors ix1: Using 8 RX queues 8 TX queues ix1: Using MSI-X interrupts with 9 vectors ix1: allocated for 8 queues ix1: allocated for 8 rx queues ix1: Ethernet address: 3c:ec:ef:e9:aa:31 ix1: eTrack 0x80000877 ix1: netmap queues/slots: TX 8/2048, RX 8/2048 pcib7: <ACPI PCI-PCI bridge> at device 23.0 on pci0 pci7: <ACPI PCI bus> on pcib7 ix2: <Intel(R) X553 (1GbE)> mem 0xdd600000-0xdd7fffff,0xdd804000-0xdd807fff= at device 0.0 on pci7 ix2: Using 2048 TX descriptors and 2048 RX descriptors ix2: Using 8 RX queues 8 TX queues ix2: Using MSI-X interrupts with 9 vectors ix2: allocated for 8 queues ix2: allocated for 8 rx queues ix2: Ethernet address: 3c:ec:ef:e9:aa:32 ix2: eTrack 0x8000087c ix2: netmap queues/slots: TX 8/2048, RX 8/2048 ix3: <Intel(R) X553 (1GbE)> mem 0xdd400000-0xdd5fffff,0xdd800000-0xdd803fff= at device 0.1 on pci7 ix3: Using 2048 TX descriptors and 2048 RX descriptors ix3: Using 8 RX queues 8 TX queues ix3: Using MSI-X interrupts with 9 vectors ix3: allocated for 8 queues ix3: allocated for 8 rx queues ix3: Ethernet address: 3c:ec:ef:e9:aa:33 ix3: eTrack 0x8000087c ix3: netmap queues/slots: TX 8/2048, RX 8/2048 pci0: <simple comms> at device 24.0 (no driver attached) isab0: <PCI-ISA bridge> at device 31.0 on pci0 isa0: <ISA bus> on isab0 pci0: <memory> at device 31.2 (no driver attached) pci0: <serial bus> at device 31.5 (no driver attached) apei0: <ACPI Platform Error Interface> on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 ns8250: UART FCR is broken uart0: console (115200,n,8,1) uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 est0: <Enhanced SpeedStep Frequency Control> on cpu0 Timecounter "TSC-low" frequency 1099999978 Hz quality 1000 Timecounters tick every 1.000 msec ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Trying to mount root from zfs:zroot/ROOT/default []... ugen0.1: <Intel XHCI root HUB> at usbus0 uhub0 on usbus0 uhub0: <Intel XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: <ST2000NM001A-2J2101 SN03> ACS-4 ATA SATA 3.x device ada0: Serial Number WS10R331 ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 1907729MB (3907029168 512 byte sectors) ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 ada1: <ST2000NM001A-2J2101 SN03> ACS-4 ATA SATA 3.x device ada1: Serial Number WS10R3FB ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 1907729MB (3907029168 512 byte sectors) ses0 at ahciem0 bus 0 scbus8 target 0 lun 0 ses0: <AHCI SGPIO Enclosure 2.00 0001> SEMB S-E-S 2.00 device ses0: SEMB SES Device ses1 at ahciem1 bus 0 scbus13 target 0 lun 0 ses1: <AHCI SGPIO Enclosure 2.00 0001> SEMB S-E-S 2.00 device ses1: SEMB SES Device ses0: ada0,pass0 in 'Slot 00', SATA Slot: scbus0 target 0 ses0: ada1,pass1 in 'Slot 01', SATA Slot: scbus1 target 0 uhub0: 8 ports with 8 removable, self powered Root mount waiting for: usbus0 usb_msc_auto_quirk: UQ_MSC_NO_GETMAXLUN set for USB mass storage device Gen= eric USB Storage (0x05e3:0x0736) Root mount waiting for: usbus0 usb_msc_auto_quirk: UQ_MSC_NO_TEST_UNIT_READY set for USB mass storage devi= ce Generic USB Storage (0x05e3:0x0736) usb_msc_auto_quirk: UQ_MSC_NO_PREVENT_ALLOW set for USB mass storage device Generic USB Storage (0x05e3:0x0736) usb_msc_auto_quirk: UQ_MSC_NO_SYNC_CACHE set for USB mass storage device Generic USB Storage (0x05e3:0x0736) ugen0.2: <Generic USB Storage> at usbus0 umass0 on uhub0 umass0: <Generic USB Storage, class 0/0, rev 2.00/2.72, addr 1> on usbus0 umass0: SCSI over Bulk-Only; quirks =3D 0xc101 umass0:14:0: Attached to scbus14 da0 at umass-sim0 bus 0 scbus14 target 0 lun 0 da0: <Generic STORAGE DEVICE 0272> Removable Direct Access SCSI device da0: Serial Number 000000000272 da0: 40.000MB/s transfers da0: 3968MB (8126464 512 byte sectors) da0: quirks=3D0x3<NO_SYNC_CACHE,NO_6_BYTE> usb_msc_auto_quirk: UQ_MSC_NO_GETMAXLUN set for USB mass storage device Gen= eric USB Storage (0x05e3:0x0736) Root mount waiting for: usbus0 usb_msc_auto_quirk: UQ_MSC_NO_TEST_UNIT_READY set for USB mass storage devi= ce Generic USB Storage (0x05e3:0x0736) usb_msc_auto_quirk: UQ_MSC_NO_PREVENT_ALLOW set for USB mass storage device Generic USB Storage (0x05e3:0x0736) usb_msc_auto_quirk: UQ_MSC_NO_SYNC_CACHE set for USB mass storage device Generic USB Storage (0x05e3:0x0736) ugen0.3: <Generic USB Storage> at usbus0 umass1 on uhub0 umass1: <Generic USB Storage, class 0/0, rev 2.00/2.72, addr 2> on usbus0 umass1: SCSI over Bulk-Only; quirks =3D 0xc101 umass1:15:1: Attached to scbus15 da1 at umass-sim1 bus 1 scbus15 target 0 lun 0 da1: <Generic STORAGE DEVICE 0272> Removable Direct Access SCSI device da1: Serial Number 000000000272 da1: 40.000MB/s transfers da1: 3968MB (8126464 512 byte sectors) da1: quirks=3D0x3<NO_SYNC_CACHE,NO_6_BYTE> ugen0.4: <vendor 0x05e3 USB2.0 Hub> at usbus0 uhub1 on uhub0 uhub1: <vendor 0x05e3 USB2.0 Hub, class 9/0, rev 2.00/32.98, addr 3> on usb= us0 uhub1: MTT enabled Root mount waiting for: usbus0 uhub1: 4 ports with 4 removable, self powered Root mount waiting for: usbus0 ugen0.5: <vendor 0x0557 product 0x7000> at usbus0 uhub2 on uhub1 uhub2: <vendor 0x0557 product 0x7000, class 9/0, rev 2.00/0.00, addr 4> on usbus0 uhub2: 4 ports with 3 removable, self powered ugen0.6: <vendor 0x0557 product 0x2419> at usbus0 ukbd0 on uhub2 ukbd0: <vendor 0x0557 product 0x2419, class 0/0, rev 1.10/1.00, addr 5> on usbus0 kbd1 at ukbd0 Root mount waiting for: usbus0 ugen0.7: <vendor 0x05e3 USB2.0 Hub> at usbus0 uhub3 on uhub1 uhub3: <vendor 0x05e3 USB2.0 Hub, class 9/0, rev 2.00/32.98, addr 6> on usb= us0 uhub3: MTT enabled uhub3: 4 ports with 4 removable, self powered Root mount waiting for: usbus0 ugen0.8: <Logitech Logitech USB Keyboard> at usbus0 ukbd1 on uhub3 ukbd1: <Logitech Logitech USB Keyboard, class 0/0, rev 1.10/28.00, addr 7> = on usbus0 kbd2 at ukbd1 Dual Console: Video Primary, Serial Secondary ichsmb0: <Intel Denverton SMBus controller> port 0xe000-0xe01f mem 0xdf478000-0xdf4780ff at device 31.4 on pci0 smbus0: <System Management Bus> on ichsmb0 lo0: link state changed to UP ums0 on uhub2 ums0: <vendor 0x0557 product 0x2419, class 0/0, rev 1.10/1.00, addr 5> on usbus0 ums0: 3 buttons and [Z] coordinates ID=3D0 ix0: link state changed to UP --=20 You are receiving this mail because: You are the assignee for the bug.=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-277389-227>