Date: Thu, 13 Mar 2008 17:57:41 -0400 From: Adam McDougall <mcdouga9@egr.msu.edu> To: stable@freebsd.org Subject: How to debug or explain slow gjournal writes? Message-ID: <47D9A355.8010102@egr.msu.edu>
next in thread | raw e-mail | index | archive | help
I am evaluating using gjournal on my servers. This one test system is running 7.0-RELEASE at the moment on a Dell PE2650 with dual 2ghz xeon, ahc0: <Adaptec aic7899 Ultra160 SCSI adapter>, and some seagate 36g 10k disks. I had the opportunity to try placing the journal consumer device on a dedicated disk. Whether or not I use a separate journal consumer device from the data consumer, I can expect the overall write speed to be slower because it is effectively writing everything twice. However when I watch it write just to the journal consumer device, the MB/sec it writes to the journal consumer device never seems to exceed around 50% of the speed that the disk is capable of. Its almost as if it is competing with IO being written to the provider, but that might be a coincidence. When I see it flush the journal consumer to the data consumer, it seems to read from a memory cache and write speed to the data consumer is full speed. Comparing with a more modern system with a mpt0: <LSILogic SAS/SATA Adapter> and two FUJITSU MAY2073RCSUN72G 0401 in a gmirror, I see journal write speed approx 40M/sec but data write speed of approx 45M/sec, which is a lot closer together, but still shows a difference. I'm fairly sure a single drive on its own could transfer faster than that, but I haven't tested it recently. I could if it would be helpful. Example with data,journal together just so I can put the terminology together with the device listing below: Writing to da2.journal shows input at approx 35M/sec and writes to the da2 journal at approx 35M/sec for a few seconds. Then the journal switches, input speed to da2.journal drops to 0, and apparently an in-memory journal gets copied to the data consumer on da2 at 65M/sec. Then it goes back to 35M/sec in and 35M/sec to the journal, repeating as expected. It seems to avoid writing to the data and journal at the same time, which is probably intentional to avoid head thrashing, especially if the journal fits in memory. Due to the slow journal write speed, and the expected double-writing of data, I only see a resulting write speed to da2.journal of approximately 16-22MB/sec. There is no improvement in the journal consumer write speed if I put it on a separate disk. The overall write speed is also the same. Additionally, when I have the journal on a gmirror (the data is too), the journal write speed is variable between 8 and 35MB/sec, it fluxuates pretty wildly second to second. As soon as I deactivate one side of the mirror, or get rid of the mirror under the journal, it is a consistent 35MB/sec. I'm looking for input on debugging, tuning, questions, bonehead errors, etc because I would like to get the most out of this setup if possible and not just settle for an inconsistent 16-22MB/sec. Thanks. Geom name: gjournal 170802896 ID: 170802896 Providers: 1. Name: da2.journal Mediasize: 35346332672 (33G) Sectorsize: 512 Mode: r1w1e1 Consumers: 1. Name: da2 Mediasize: 36420075008 (34G) Sectorsize: 512 Mode: r1w1e1 Jend: 36420074496 Jstart: 35346332672 Role: Data,Journal
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?47D9A355.8010102>