From owner-freebsd-stable@FreeBSD.ORG Tue Oct 10 17:55:10 2006 Return-Path: X-Original-To: stable@freebsd.org Delivered-To: freebsd-stable@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6AE7016A40F for ; Tue, 10 Oct 2006 17:55:10 +0000 (UTC) (envelope-from dmitry@atlantis.dp.ua) Received: from postman.atlantis.dp.ua (postman.atlantis.dp.ua [193.108.47.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7B3CC43D73 for ; Tue, 10 Oct 2006 17:55:06 +0000 (GMT) (envelope-from dmitry@atlantis.dp.ua) Received: from atlantis.dp.ua (localhost [127.0.0.1]) by postman.atlantis.dp.ua (8.13.1/8.13.1) with ESMTP id k9AHskJH004402; Tue, 10 Oct 2006 20:54:56 +0300 (EEST) (envelope-from dmitry@atlantis.dp.ua) Received: from localhost (dmitry@localhost) by atlantis.dp.ua (8.13.1/8.13.1/Submit) with ESMTP id k9AHsk78004399; Tue, 10 Oct 2006 20:54:46 +0300 (EEST) (envelope-from dmitry@atlantis.dp.ua) Date: Tue, 10 Oct 2006 20:54:46 +0300 (EEST) From: Dmitry Pryanishnikov To: Vivek Khera In-Reply-To: <71E7CE6F-900A-419D-84C0-18B926377AED@khera.org> Message-ID: <20061010203856.K68266@atlantis.atlantis.dp.ua> References: <71E7CE6F-900A-419D-84C0-18B926377AED@khera.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: stable@freebsd.org Subject: Re: gmirror warnings X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Oct 2006 17:55:10 -0000 Hello! On Tue, 10 Oct 2006, Vivek Khera wrote: > Over the weekend and today we got some curious error messages from gmirror > providers: > > Oct 7 03:02:14 dtfe2 kernel: ad6: FAILURE - out of memory in start > Oct 7 04:15:58 dtfe2 kernel: ad4: FAILURE - out of memory in start > > Should I be worried about these warnings or are they just transient failures > that correct themselves? Those messages are not directly related to gmirror, they come from the ATA driver when it can't allocate memory for the structure which describes I/O request: ata-disk.c: static void ad_strategy(struct bio *bp) { device_t dev = bp->bio_disk->d_drv1; struct ata_device *atadev = device_get_softc(dev); struct ata_request *request; if (!(request = ata_alloc_request())) { device_printf(dev, "FAILURE - out of memory in start\n"); biofinish(bp, NULL, ENOMEM); return; } ata-all.h: #define ata_alloc_request() uma_zalloc(ata_request_zone, M_NOWAIT | M_ZERO) Yes, message is somewhat confusing. Those situations will cause original I/O request to fail with errno = ENOMEM. The cure for the problem is proper tuning of the KVA-related values - too bad tuning(7) doesn't even mention KVA or kmem. Off the memory, lowering this: dmitry@homelynx$ sysctl vm.kmem_size_scale vm.kmem_size_scale: 3 from the /boot/loader.conf, e.g.: vm.kmem_size_scale=2 will give you more kernel virtual memory to burn. I'm sure that there are more options, but they aren't documented ;( Sincerely, Dmitry -- Atlantis ISP, System Administrator e-mail: dmitry@atlantis.dp.ua nic-hdl: LYNX-RIPE