Date: Sun, 14 Sep 2008 10:26:19 GMT From: Ruslan <yalur2mail.ru@FreeBSD.org> To: freebsd-gnats-submit@FreeBSD.org Subject: i386/127375: If vm.kmem_size_max>"1073741823" then write speed to ZFS pool decrease 3-5 times. Message-ID: <200809141026.m8EAQJ3m032741@www.freebsd.org> Resent-Message-ID: <200809141030.m8EAU2Ht062052@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 127375
>Category: i386
>Synopsis: If vm.kmem_size_max>"1073741823" then write speed to ZFS pool decrease 3-5 times.
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: freebsd-i386
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Sun Sep 14 10:30:02 UTC 2008
>Closed-Date:
>Last-Modified:
>Originator: Ruslan
>Release: 7.0-RELEASE-p3
>Organization:
Home use
>Environment:
FreeBSD yalur.monolit.ntu-kpi.kiev.ua 7.0-RELEASE-p3 FreeBSD 7.0-RELEASE-p3 #51: Sun Aug 31 17:42:15 EEST 2008 root@yalur.monolit.ntu-kpi.kiev.ua:/usr/src/sys/i386/compile/MYKERNEL i386
>Description:
This problem related only with i386 and was not observed on amd64. I found this problem since FreeBSD 7.0 RC2 have used first time.
If I change vm.kmem_size* to
vm.kmem_size="1G" or more
vm.kmem_size_max="1G" or more
KVA_PAGES=512
Write speed is 30-40 Mb/sec.
[root@yalur /home/ruslan]# dd if=/dev/zero of=/mnt/Temp/22 bs=1m
963+0 records in
963+0 records out
1008730112 bytes transferred in 29.295576 secs (34432848 bytes/sec)
If I chenge vm.kmem_size* to 1G minus 1byte:
vm.kmem_size_max="1073741823"
vm.kmem_size="1073741823"
KVA_PAGES=512
Write speed is 130-140 Mb/sec.
[root@yalur /home/ruslan]# dd if=/dev/zero of=/mnt/Temp/22 bs=1m
963+0 records in
963+0 records out
1008730112 bytes transferred in 7.550406 secs (133567341 bytes/sec)
Zpool status:
[root@yalur /home/ruslan]# zpool status
pool: data
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
data ONLINE 0 0 0
raidz1 ONLINE 0 0 0
ad6 ONLINE 0 0 0
ad4 ONLINE 0 0 0
ad9 ONLINE 0 0 0
ad12 ONLINE 0 0 0
raidz1 ONLINE 0 0 0
ad11 ONLINE 0 0 0
ad10 ONLINE 0 0 0
ad8 ONLINE 0 0 0
ad16 ONLINE 0 0 0
errors: No known data errors
This problem not related whith hardware. I observed this problem on motherboards:
Asus M2N-SLI Deluxe and Sapphire PC-AM2RD790. Alsow I used different HDDs - resault is the same.
[root@yalur /home/ruslan]# dmesg
Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 7.0-RELEASE-p3 #51: Sun Aug 31 17:42:15 EEST 2008
root@yalur.monolit.ntu-kpi.kiev.ua:/usr/src/sys/i386/compile/MYKERNEL
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 4400+ (2760.21-MHz 686-class CPU)
Origin = "AuthenticAMD" Id = 0x60fb2 Stepping = 2
Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
Features2=0x2001<SSE3,CX16>
AMD Features=0xea500800<SYSCALL,NX,MMX+,FFXSR,RDTSCP,LM,3DNow!+,3DNow!>
AMD Features2=0x11f<LAHF,CMP,SVM,ExtAPIC,CR8,Prefetch>
Cores per package: 2
real memory = 3489529856 (3327 MB)
avail memory = 3411312640 (3253 MB)
ACPI APIC Table: <RD790 AWRDACPI>
AP #1 (PHY# 1) failed!
panic y/n? [y] FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
ioapic0: Changing APIC ID to 4
ioapic0 <Version 2.1> irqs 0-23 on motherboard
bktr_mem: memory holder loaded
kbd1 at kbdmux0
kqemu version 0x00010300
kqemu: KQEMU installed, max_locked_mem=1699484kB.
acpi0: <RD790 AWRDACPI> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
ACPI Error (uteval-0322): No object was returned from [\\_SB_.PCI0.SMB0._CRS] (Node 0x8649d860), AE_NOT_EXIST
can't fetch resources for \\_SB_.PCI0.SMB0 - AE_NOT_EXIST
acpi0: reservation of 0, cffe0000 (3) failed
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0
acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
device_attach: acpi_hpet0 attach returned 12
cpu0: <ACPI CPU> on acpi0
powernow0: <PowerNow! K8> on cpu0
cpu1: <ACPI CPU> on acpi0
powernow1: <PowerNow! K8> on cpu1
acpi_button0: <Power Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
pci1: <ACPI PCI bus> on pcib1
vgapci0: <VGA-compatible display> port 0xbe00-0xbeff mem 0xd0000000-0xdfffffff,0xfddf0000-0xfddfffff irq 18 at device 0.0 on pci1
vgapci1: <VGA-compatible display> mem 0xfdde0000-0xfddeffff at device 0.1 on pci1
pcib2: <ACPI PCI-PCI bridge> at device 4.0 on pci0
pci2: <ACPI PCI bus> on pcib2
mskc0: <D-Link 560T Gigabit Ethernet> port 0xae00-0xaeff mem 0xfdcfc000-0xfdcfffff irq 16 at device 0.0 on pci2
msk0: <Marvell Technology Group Ltd. Yukon EC Id 0xb6 Rev 0x02> on mskc0
msk0: Ethernet address: 00:1b:11:79:46:b7
miibus0: <MII bus> on msk0
e1000phy0: <Marvell 88E1111 Gigabit PHY> PHY 0 on miibus0
e1000phy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto
mskc0: [FILTER]
pcib3: <ACPI PCI-PCI bridge> at device 7.0 on pci0
pci3: <ACPI PCI bus> on pcib3
mykc0: <Generic Marvell Yukon 88E8053 based Ethernet Controller Ver. 10.51.2.3> port 0xee00-0xeeff mem 0xfdafc000-0xfdafffff irq 19 at device 0.0 on pci3
myk0 on mykc0
myk0: Ethernet address: 00:01:29:a3:a6:df
mykc0: [ITHREAD]
pcib4: <ACPI PCI-PCI bridge> at device 9.0 on pci0
pci4: <ACPI PCI bus> on pcib4
mykc1: <Generic Marvell Yukon 88E8052 based Ethernet Controller Ver. 10.51.2.3> port 0xde00-0xdeff mem 0xfd8fc000-0xfd8fffff irq 17 at device 0.0 on pci4
myk1 on mykc1
myk1: Ethernet address: 00:01:29:a3:a7:11
mykc1: [ITHREAD]
pcib5: <ACPI PCI-PCI bridge> at device 10.0 on pci0
pci5: <ACPI PCI bus> on pcib5
atapci0: <SiI SiI 3132 SATA300 controller> port 0xcf00-0xcf7f mem 0xfd6ff000-0xfd6ff07f,0xfd6f8000-0xfd6fbfff irq 18 at device 0.0 on pci5
atapci0: [ITHREAD]
ata2: <ATA channel 0> on atapci0
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci0
ata3: [ITHREAD]
atapci1: <ATI ATA controller> port 0xff00-0xff07,0xfe00-0xfe03,0xfd00-0xfd07,0xfc00-0xfc03,0xfb00-0xfb0f mem 0xfdfff000-0xfdfff3ff irq 22 at device 18.0 on pci0
atapci1: [ITHREAD]
ata4: <ATA channel 0> on atapci1
ata4: [ITHREAD]
ata5: <ATA channel 1> on atapci1
ata5: [ITHREAD]
ohci0: <OHCI (generic) USB controller> mem 0xfdffe000-0xfdffefff irq 16 at device 19.0 on pci0
ohci0: [GIANT-LOCKED]
ohci0: [ITHREAD]
usb0: OHCI version 1.0, legacy support
usb0: SMM does not respond, resetting
usb0: <OHCI (generic) USB controller> on ohci0
usb0: USB revision 1.0
uhub0: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0
uhub0: 2 ports with 2 removable, self powered
ohci1: <OHCI (generic) USB controller> mem 0xfdffd000-0xfdffdfff irq 17 at device 19.1 on pci0
ohci1: [GIANT-LOCKED]
ohci1: [ITHREAD]
usb1: OHCI version 1.0, legacy support
usb1: SMM does not respond, resetting
usb1: <OHCI (generic) USB controller> on ohci1
usb1: USB revision 1.0
uhub1: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb1
uhub1: 2 ports with 2 removable, self powered
ohci2: <OHCI (generic) USB controller> mem 0xfdffc000-0xfdffcfff irq 18 at device 19.2 on pci0
ohci2: [GIANT-LOCKED]
ohci2: [ITHREAD]
usb2: OHCI version 1.0, legacy support
usb2: SMM does not respond, resetting
usb2: <OHCI (generic) USB controller> on ohci2
usb2: USB revision 1.0
uhub2: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb2
uhub2: 2 ports with 2 removable, self powered
ohci3: <OHCI (generic) USB controller> mem 0xfdffb000-0xfdffbfff irq 17 at device 19.3 on pci0
ohci3: [GIANT-LOCKED]
ohci3: [ITHREAD]
usb3: OHCI version 1.0, legacy support
usb3: SMM does not respond, resetting
usb3: <OHCI (generic) USB controller> on ohci3
usb3: USB revision 1.0
uhub3: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb3
uhub3: 2 ports with 2 removable, self powered
ohci4: <OHCI (generic) USB controller> mem 0xfdffa000-0xfdffafff irq 18 at device 19.4 on pci0
ohci4: [GIANT-LOCKED]
ohci4: [ITHREAD]
usb4: OHCI version 1.0, legacy support
usb4: SMM does not respond, resetting
usb4: <OHCI (generic) USB controller> on ohci4
usb4: USB revision 1.0
uhub4: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb4
uhub4: 2 ports with 2 removable, self powered
ehci0: <EHCI (generic) USB 2.0 controller> mem 0xfdff9000-0xfdff90ff irq 19 at device 19.5 on pci0
ehci0: [GIANT-LOCKED]
ehci0: [ITHREAD]
usb5: EHCI version 1.0
usb5: companion controllers, 2 ports each: usb0 usb1 usb2 usb3 usb4
usb5: <EHCI (generic) USB 2.0 controller> on ehci0
usb5: USB revision 2.0
uhub5: <ATI EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb5
uhub5: 10 ports with 10 removable, self powered
pci0: <serial bus, SMBus> at device 20.0 (no driver attached)
atapci2: <ATI IXP600 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xf900-0xf90f at device 20.1 on pci0
ata0: <ATA channel 0> on atapci2
ata0: [ITHREAD]
isab0: <PCI-ISA bridge> at device 20.3 on pci0
isa0: <ISA bus> on isab0
pcib6: <ACPI PCI-PCI bridge> at device 20.4 on pci0
pci6: <ACPI PCI bus> on pcib6
atapci3: <Promise PDC40719 SATA300 controller> port 0x9f00-0x9f7f,0x9a00-0x9aff mem 0xfd5ff000-0xfd5fffff,0xfd5c0000-0xfd5dffff irq 20 at device 5.0 on pci6
atapci3: [ITHREAD]
atapci3: [ITHREAD]
ata6: <ATA channel 0> on atapci3
ata6: [ITHREAD]
ata7: <ATA channel 1> on atapci3
ata7: [ITHREAD]
ata8: <ATA channel 2> on atapci3
ata8: [ITHREAD]
ata9: <ATA channel 3> on atapci3
ata9: [ITHREAD]
bktr0: <BrookTree 878> mem 0xfd4ff000-0xfd4fffff irq 21 at device 6.0 on pci6
bktr0: [GIANT-LOCKED]
bktr0: [ITHREAD]
bktr0: AVer Media TV/FM, Philips FR1216 PAL FM tuner.
pci6: <multimedia> at device 6.1 (no driver attached)
pcm0: <Creative Audigy 2 (EMU10K2)> port 0x9e00-0x9e3f irq 22 at device 7.0 on pci6
pcm0: <SigmaTel STAC9721/23 AC97 Codec>
pcm0: [ITHREAD]
fwohci0: <1394 Open Host Controller Interface> mem 0xfd5fe000-0xfd5fe7ff,0xfd5f8000-0xfd5fbfff irq 23 at device 7.2 on pci6
fwohci0: [FILTER]
fwohci0: OHCI version 1.10 (ROM=0)
fwohci0: No. of Isochronous channels is 4.
fwohci0: EUI64 00:02:3c:01:01:04:70:b1
fwohci0: Phy 1394a available S400, 2 ports.
fwohci0: Link S400, max_rec 2048 bytes.
firewire0: <IEEE1394(FireWire) bus> on fwohci0
fwe0: <Ethernet over FireWire> on firewire0
if_fwe0: Fake Ethernet address: 02:02:3c:04:70:b1
fwe0: Ethernet address: 02:02:3c:04:70:b1
fwip0: <IP over FireWire> on firewire0
fwip0: Firewire address: 00:02:3c:01:01:04:70:b1 @ 0xfffe00000000, S400, maxrec 2048
sbp0: <SBP-2/SCSI over FireWire> on firewire0
fwohci0: Initiate bus reset
fwohci0: BUS reset
fwohci0: node_id=0xc800ffc0, gen=1, CYCLEMASTER mode
fwohci1: <VIA Fire II (VT6306)> port 0x9c00-0x9c7f mem 0xfd5fd000-0xfd5fd7ff irq 23 at device 8.0 on pci6
fwohci1: [FILTER]
fwohci1: OHCI version 1.10 (ROM=1)
fwohci1: No. of Isochronous channels is 4.
fwohci1: EUI64 00:01:29:20:00:06:cc:52
fwohci1: Phy 1394a available S400, 2 ports.
fwohci1: Link S400, max_rec 2048 bytes.
firewire1: <IEEE1394(FireWire) bus> on fwohci1
fwe1: <Ethernet over FireWire> on firewire1
if_fwe1: Fake Ethernet address: 02:01:29:06:cc:52
fwe1: Ethernet address: 02:01:29:06:cc:52
fwip1: <IP over FireWire> on firewire1
fwip1: Firewire address: 00:01:29:20:00:06:cc:52 @ 0xfffe00000000, S400, maxrec 2048
sbp1: <SBP-2/SCSI over FireWire> on firewire1
fwohci1: Initiate bus reset
fwohci1: BUS reset
fwohci1: node_id=0xc800ffc0, gen=1, CYCLEMASTER mode
acpi_tz0: <Thermal Zone> on acpi0
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: [FILTER]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
sio0: configured irq 4 not in bitmap of probed irqs 0
sio0: port may not be enabled
sio0: configured irq 4 not in bitmap of probed irqs 0
sio0: port may not be enabled
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
sio0: type 16550A
sio0: [FILTER]
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
device_attach: acpi_hpet0 attach returned 12
pmtimer0 on isa0
orm0: <ISA Option ROM> at iomem 0xd9800-0xdd7ff pnpid ORM0000 on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: parallel port not found.
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
uhub0: device problem (TIMEOUT), disabling port 1
WARNING: ZFS is considered to be an experimental feature in FreeBSD.
Timecounters tick every 1.000 msec
ipfw2 initialized, divert enabled, rule-based forwarding disabled, default to deny, logging limited to 10 packets/entry by default
firewire0: 1 nodes, maxhop <= 0, cable IRM = 0 (me)
firewire0: bus manager 0 (me)
firewire1: 1 nodes, maxhop <= 0, cable IRM = 0 (me)
firewire1: bus manager 0 (me)
ZFS filesystem version 6
ZFS storage pool version 6
acd0: CDRW <CD-W540E/1.0D> at ata0-master UDMA33
acd1: DVDR <NEC DVD RW ND-4571A/1-01> at ata0-slave UDMA33
ad4: 476940MB <Seagate ST3500320AS SD15> at ata2-master SATA300
ad6: 476940MB <MAXTOR STM3500320AS MX15> at ata3-master SATA300
ad8: 305245MB <Seagate ST3320620AS 3.AAD> at ata4-master UDMA33
ad9: 476940MB <Seagate ST3500320AS SD15> at ata4-slave UDMA33
ad10: 305245MB <Seagate ST3320620AS 3.AAG> at ata5-master UDMA33
ad11: 305245MB <Seagate ST3320620AS 3.AAG> at ata5-slave UDMA33
ad12: 476940MB <MAXTOR STM3500320AS MX15> at ata6-master SATA300
ad14: 76319MB <MAXTOR STM380815AS 4.AAB> at ata7-master SATA300
ad16: 305245MB <Seagate ST3320620AS 3.AAD> at ata8-master SATA300
ad18: 76319MB <Seagate ST380815AS 3.AAD> at ata9-master SATA300
GEOM_MIRROR: Device mirror/gm0 launched (2/2).
cd0 at ata0 bus 0 target 0 lun 0
cd0: <TEAC CD-W540E 1.0D> Removable CD-ROM SCSI-0 device
cd0: 33.000MB/s transfers
cd0: Attempt to query device size failed: NOT READY, Medium not present
cd1 at ata0 bus 0 target 1 lun 0
cd1: <_NEC DVD_RW ND-4571A 1-01> Removable CD-ROM SCSI-0 device
cd1: 33.000MB/s transfers
cd1: Attempt to query device size failed: NOT READY, Medium not present
SMP: AP CPU #1 Launched!
Trying to mount root from ufs:/dev/mirror/gm0s2a
internet0: link up
home1: link up
fuse4bsd: version 0.3.9-pre1, FUSE ABI 7.8
ums0: <A4Tech PS/2+USB Mouse, class 0/0, rev 1.10/0.01, addr 2> on uhub1
ums0: 7 buttons and Z dir.
tap0: Ethernet address: 00:bd:d1:f6:16:00
tap1: Ethernet address: 00:bd:5d:89:17:01
home1: link down
2 Pair Downshift detectedhome1: link up
home1: link down
home1: link up
home1: link down
home1: link up
home1: link down
home1: link up
>How-To-Repeat:
Use i386 FreeBSD 7.0 p1,p2,p3. Change vm.kmem_size* to 1G. check (KVA_PAGES=512):
dd if=/dev/zero of=/zfs_pool/file1 bs=1m. Then chenge vm.kmem_size* to 1G minus 1byte (KVA_PAGES=512) and check dd if=/dev/zero of=/zfs_pool/file2 bs=1m.
>Fix:
No.
>Release-Note:
>Audit-Trail:
>Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200809141026.m8EAQJ3m032741>