From owner-freebsd-fs@FreeBSD.ORG Mon Apr 7 01:32:18 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6A47037B401; Mon, 7 Apr 2003 01:32:18 -0700 (PDT) Received: from obsecurity.dyndns.org (adsl-63-207-60-150.dsl.lsan03.pacbell.net [63.207.60.150]) by mx1.FreeBSD.org (Postfix) with ESMTP id D038F43FA3; Mon, 7 Apr 2003 01:32:17 -0700 (PDT) (envelope-from kris@obsecurity.org) Received: from rot13.obsecurity.org (rot13.obsecurity.org [10.0.0.5]) by obsecurity.dyndns.org (Postfix) with ESMTP id 9AD4166D16; Mon, 7 Apr 2003 01:32:17 -0700 (PDT) Received: by rot13.obsecurity.org (Postfix, from userid 1000) id 8F34B447; Mon, 7 Apr 2003 01:32:17 -0700 (PDT) Date: Mon, 7 Apr 2003 01:32:17 -0700 From: Kris Kennaway To: fs@FreeBSD.org, kirk@mckusick.com, jeff@FreeBSD.org Message-ID: <20030407083217.GA56068@rot13.obsecurity.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="IJpNTDwzlM2Ie8A6" Content-Disposition: inline User-Agent: Mutt/1.4i Subject: "panic: sleeping thread owns a mutex" in bremfree() X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2003 08:32:18 -0000 --IJpNTDwzlM2Ie8A6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline I just got the following panic on alpha: panic: sleeping thread owns a mutex propagate_priority() at propagate_priority+0x148 _mtx_lock_sleep() at _mtx_lock_sleep+0x264 _mtx_lock_flags() at _mtx_lock_flags+0x84 bremfree() at bremfree+0x30 vfs_backgroundwritedone() at vfs_backgroundwritedone+0x1c8 bufdone() at bufdone+0x1c4 bufdonebio() at bufdonebio+0x1c biodone() at biodone+0x90 g_dev_done() at g_dev_done+0xd8 biodone() at biodone+0x90 g_io_schedule_up() at g_io_schedule_up+0x128 g_up_procbody() at g_up_procbody+0x5c fork_exit() at fork_exit+0x100 exception_return() at exception_return --- root of call graph --- As usual, no gdb backtrace possible because it's broken on alpha. Kris --IJpNTDwzlM2Ie8A6 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (FreeBSD) iD8DBQE+kTeRWry0BWjoQKURAreDAKC5V2xhcByUx2wTZhyl2bG1tww+ygCfTDq2 ERZPw8tNUz4KcxJHm6fz7H0= =Kda4 -----END PGP SIGNATURE----- --IJpNTDwzlM2Ie8A6-- From owner-freebsd-fs@FreeBSD.ORG Wed Apr 9 02:59:33 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8B43A37B401 for ; Wed, 9 Apr 2003 02:59:33 -0700 (PDT) Received: from pierce.numericable.net (pierce.numericable.net [80.236.0.150]) by mx1.FreeBSD.org (Postfix) with SMTP id 4A66943FA3 for ; Wed, 9 Apr 2003 02:59:32 -0700 (PDT) (envelope-from brice@mouarf.net) Received: (qmail 27227 invoked from network); 9 Apr 2003 09:59:30 -0000 Received: from unknown (HELO localhost.localdomain) ([80.236.121.31]) (envelope-sender ) by 0 (qmail-ldap-1.03) with SMTP for ; 9 Apr 2003 09:59:30 -0000 Received: from localhost.localdomain (localhost [127.0.0.1]) by localhost.localdomain (8.12.9/8.12.9) with ESMTP id h39A22sP009858 for ; Wed, 9 Apr 2003 12:02:02 +0200 (CEST) (envelope-from brice@mouarf.net) Received: (from bgensburger@localhost) by localhost.localdomain (8.12.9/8.12.9/Submit) id h39A21Uj009857 for freebsd-fs@freebsd.org; Wed, 9 Apr 2003 12:02:01 +0200 (CEST) (envelope-from brice@mouarf.net) X-Authentication-Warning: localhost.localdomain: bgensburger set sender to brice@mouarf.net using -f Date: Wed, 9 Apr 2003 12:02:01 +0200 From: Brice Gensburger To: freebsd-fs@freebsd.org Message-ID: <20030409100201.GA9628@localhost.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.1i Subject: Mounting a msdos-formated USB Mass storage device X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2003 09:59:33 -0000 (mail sent to freebsd-questions, but maybe i'll have better luck here?) Hello, I'm having a bit of a hard time with an external USB2.0/Firewire HD. [localhost][~]$ uname -a FreeBSD localhost.localdomain 4.8-STABLE FreeBSD 4.8-STABLE #2: Mon Apr 7 12:21:33 CEST 2003 root@localhost.localdomain:/usr/src/sys/compile/IPSEC i386 (was CVSup'ed just before compile) The problem doesn't seem to be in the USB part: the modules are OK, the drive is recognized OK: from dmesg: uhci0: port 0xfce0-0xfcff irq 11 at device 4.2 on pci0 usb0: on uhci0 usb0: USB revision 1.0 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered umass0: DMI USB 2.0 Storage Adaptor, rev 2.00/11.10, addr 2 (...) da0 at umass-sim0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-0 device da0: 650KB/s transfers da0: 117246MB (240121728 512 byte sectors: 64H 32S/T 51710C) (da0:umass-sim0:0:0:0): READ(6)/WRITE(6) not supported, increasing minimum_cmd_size to 10. Disconnection/Reconnection works OK umass0: at uhub0 port 2 (addr 2) disconnected (da0:umass-sim0:0:0:0): lost device (da0:umass-sim0:0:0:0): removing device entry umass0: detached umass0: DMI USB 2.0 Storage Adaptor, rev 2.00/11.10, addr 2 da0 at umass-sim0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-0 device da0: 650KB/s transfers da0: 117246MB (240121728 512 byte sectors: 64H 32S/T 51710C) (da0:umass-sim0:0:0:0): READ(6)/WRITE(6) not supported, increasing minimum_cmd_size to 10. The CAM Part seems to work OK too: # camcontrol inquiry 0:0 pass0: Fixed Direct Access SCSI-0 device pass0: Serial Number pass0: 650KB/s transfers my problem is when i try to mount it. It's already formatted (Win)(yeah, yeah, but i want to be able to use it on windows boxes, unices and maybe MacOSX ones..). fdisk /dev/da0 ******* Working on device /dev/da0 ******* parameters extracted from in-core disklabel are: cylinders=14946 heads=255 sectors/track=63 (16065 blks/cyl) Figures below won't work with BIOS for partitions not in cyl 1 parameters to be used for BIOS calculations are: cylinders=14946 heads=255 sectors/track=63 (16065 blks/cyl) Media sector size is 512 Warning: BIOS sector numbering starts with sector 1 Information from DOS bootblock is: The data for partition 1 is: sysid 15,(Extended DOS, LBA) start 16065, size 240091425 (117232 Meg), flag 0 beg: cyl 1/ head 0/ sector 1; end: cyl 1023/ head 254/ sector 63 The data for partition 2 is: The data for partition 3 is: The data for partition 4 is: disklabel /dev/da0 # /dev/da0: type: SCSI disk: Maxtor 6 label: Y120P0 flags: bytes/sector: 512 sectors/track: 63 tracks/cylinder: 255 sectors/cylinder: 16065 cylinders: 14946 sectors/unit: 240121728 rpm: 3600 interleave: 1 trackskew: 0 cylinderskew: 0 headswitch: 0 # milliseconds track-to-track seek: 0 # milliseconds drivedata: 0 8 partitions: # size offset fstype [fsize bsize bps/cpg] c: 240121728 0 unused 0 0 # (Cyl. 0 - 14946*) ok, let's try to mount that # mount /dev/da0s1c /usbdrive mount: /dev/da0s1c on /usbdrive: incorrect super block which is only natural, since it's not UFS. # mount_msdos /dev/da0s1c /usbdrive mount_msdos: /dev/da0s1c: Invalid argument I googled for this, and found several posts saying to try to mount the 4th slice for Zip Drives and such. Ok, let's try that: # mount_msdos /dev/da0s4 /usbdrive mount_msdos: /dev/da0s4: Invalid argument Mike Meyer told me i had to mount /dev/da0s1: # mount -t msdos /dev/da0s1 /usbdrive msdos: /dev/da0s1: Invalid argument so, doesn't work better. While searching, i found how to extract additional info (gpart): # gpart -v -d /dev/da0 dev(/dev/da0) mss(512) chs(14946/255/63)(LBA) #s(240107490) size(117239mb) Primary partition(1) type: 015(0x0F)(Extended DOS, LBA) size: 117232mb #s(240091425) s(16065-240107489) chs: (1/0/1)-(1023/254/63)d (1/0/1)-(14945/254/63)r hex: 00 00 01 01 0F FE FF FF C1 3E 00 00 21 81 4F 0E Logical partition type: 011(0x0B)(DOS or Windows 95 with 32 bit FAT) size: 117232mb #s(240091362) s(16128-240107489) chs: (1/1/1)-(1023/254/63)d (1/1/1)-(14945/254/63)r hex: 00 01 01 01 0B FE FF FF 3F 00 00 00 E2 80 4F 0E (others are unused) Which leads me to think it's a geometry problem.. the CHS reported by gpart is 14946/255/63, while in messages i have: Apr 9 10:27:01 localhost /kernel: da0: 117246MB (240121728 512 byte sectors: 64H 32S/T 51710C) Any idea?? From owner-freebsd-fs@FreeBSD.ORG Wed Apr 9 04:24:57 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F000837B404 for ; Wed, 9 Apr 2003 04:24:57 -0700 (PDT) Received: from thomas.numericable.net (thomas.numericable.net [80.236.0.149]) by mx1.FreeBSD.org (Postfix) with SMTP id 6750743FB1 for ; Wed, 9 Apr 2003 04:24:56 -0700 (PDT) (envelope-from brice@mouarf.net) Received: (qmail 15792 invoked from network); 9 Apr 2003 11:24:54 -0000 Received: from unknown (HELO localhost.localdomain) ([80.236.121.31]) (envelope-sender ) by 0 (qmail-ldap-1.03) with SMTP for ; 9 Apr 2003 11:24:54 -0000 Received: from localhost.localdomain (localhost [127.0.0.1]) by localhost.localdomain (8.12.9/8.12.9) with ESMTP id h39BRPsP010342 for ; Wed, 9 Apr 2003 13:27:25 +0200 (CEST) (envelope-from brice@mouarf.net) Received: (from bgensburger@localhost) by localhost.localdomain (8.12.9/8.12.9/Submit) id h39BRPgH010341 for freebsd-fs@freebsd.org; Wed, 9 Apr 2003 13:27:25 +0200 (CEST) (envelope-from brice@mouarf.net) X-Authentication-Warning: localhost.localdomain: bgensburger set sender to brice@mouarf.net using -f Date: Wed, 9 Apr 2003 13:27:25 +0200 From: Brice Gensburger To: freebsd-fs@freebsd.org Message-ID: <20030409112725.GA10219@localhost.localdomain> References: <20030409100201.GA9628@localhost.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030409100201.GA9628@localhost.localdomain> User-Agent: Mutt/1.5.1i Subject: Re: Mounting a msdos-formated USB Mass storage device X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2003 11:24:58 -0000 Got it ! Thanks to Crist J. Clark and Google, i finally found the relevant info... http://www.geocrawler.com/mail/msg.php3?msg_id=825592&list=150 Long made short: # /dev/MAKEDEV da0s5 and then, mounting da0s5.. It's an extended DOS partition, and those seem to behave differently regarding naming. Thanks for your time :-) On Wed, Apr 09, 2003 at 12:02:01PM +0200, Brice Gensburger dared to send a mail and wrote: > (mail sent to freebsd-questions, but maybe i'll have better luck here?) > > Hello, > > I'm having a bit of a hard time with an external USB2.0/Firewire HD. > > [localhost][~]$ uname -a > FreeBSD localhost.localdomain 4.8-STABLE FreeBSD 4.8-STABLE #2: Mon Apr 7 12:21:33 CEST 2003 root@localhost.localdomain:/usr/src/sys/compile/IPSEC i386 > > (was CVSup'ed just before compile) > > > The problem doesn't seem to be in the USB part: the modules are OK, the drive is recognized OK: > > from dmesg: > > uhci0: port 0xfce0-0xfcff irq 11 at device 4.2 on pci0 > usb0: on uhci0 > usb0: USB revision 1.0 > uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 > uhub0: 2 ports with 2 removable, self powered > umass0: DMI USB 2.0 Storage Adaptor, rev 2.00/11.10, addr 2 > > (...) > da0 at umass-sim0 bus 0 target 0 lun 0 > da0: Fixed Direct Access SCSI-0 device > da0: 650KB/s transfers > da0: 117246MB (240121728 512 byte sectors: 64H 32S/T 51710C) > (da0:umass-sim0:0:0:0): READ(6)/WRITE(6) not supported, increasing minimum_cmd_size to 10. > > Disconnection/Reconnection works OK > > umass0: at uhub0 port 2 (addr 2) disconnected > (da0:umass-sim0:0:0:0): lost device > (da0:umass-sim0:0:0:0): removing device entry > umass0: detached > umass0: DMI USB 2.0 Storage Adaptor, rev 2.00/11.10, addr 2 > da0 at umass-sim0 bus 0 target 0 lun 0 > da0: Fixed Direct Access SCSI-0 device > da0: 650KB/s transfers > da0: 117246MB (240121728 512 byte sectors: 64H 32S/T 51710C) > (da0:umass-sim0:0:0:0): READ(6)/WRITE(6) not supported, increasing minimum_cmd_size to 10. > > The CAM Part seems to work OK too: > # camcontrol inquiry 0:0 > pass0: Fixed Direct Access SCSI-0 device > pass0: Serial Number > pass0: 650KB/s transfers > > my problem is when i try to mount it. It's already formatted (Win)(yeah, yeah, but i want to be able to use it on windows boxes, unices and maybe MacOSX ones..). > > fdisk /dev/da0 > ******* Working on device /dev/da0 ******* > parameters extracted from in-core disklabel are: > cylinders=14946 heads=255 sectors/track=63 (16065 blks/cyl) > > Figures below won't work with BIOS for partitions not in cyl 1 > parameters to be used for BIOS calculations are: > cylinders=14946 heads=255 sectors/track=63 (16065 blks/cyl) > > Media sector size is 512 > Warning: BIOS sector numbering starts with sector 1 > Information from DOS bootblock is: > The data for partition 1 is: > sysid 15,(Extended DOS, LBA) > start 16065, size 240091425 (117232 Meg), flag 0 > beg: cyl 1/ head 0/ sector 1; > end: cyl 1023/ head 254/ sector 63 > The data for partition 2 is: > > The data for partition 3 is: > > The data for partition 4 is: > > > disklabel /dev/da0 > # /dev/da0: > type: SCSI > disk: Maxtor 6 > label: Y120P0 > flags: > bytes/sector: 512 > sectors/track: 63 > tracks/cylinder: 255 > sectors/cylinder: 16065 > cylinders: 14946 > sectors/unit: 240121728 > rpm: 3600 > interleave: 1 > trackskew: 0 > cylinderskew: 0 > headswitch: 0 # milliseconds > track-to-track seek: 0 # milliseconds > drivedata: 0 > > 8 partitions: > # size offset fstype [fsize bsize bps/cpg] > c: 240121728 0 unused 0 0 # (Cyl. 0 - 14946*) > > ok, let's try to mount that > > # mount /dev/da0s1c /usbdrive > mount: /dev/da0s1c on /usbdrive: incorrect super block > > which is only natural, since it's not UFS. > > # mount_msdos /dev/da0s1c /usbdrive > mount_msdos: /dev/da0s1c: Invalid argument > > I googled for this, and found several posts saying to try to mount the 4th slice for Zip Drives and such. Ok, let's try that: > > # mount_msdos /dev/da0s4 /usbdrive > mount_msdos: /dev/da0s4: Invalid argument > > Mike Meyer told me i had to mount /dev/da0s1: > > # mount -t msdos /dev/da0s1 /usbdrive > msdos: /dev/da0s1: Invalid argument > > so, doesn't work better. > While searching, i found how to extract additional info (gpart): > > # gpart -v -d /dev/da0 > > dev(/dev/da0) mss(512) chs(14946/255/63)(LBA) #s(240107490) size(117239mb) > Primary partition(1) > type: 015(0x0F)(Extended DOS, LBA) > size: 117232mb #s(240091425) s(16065-240107489) > chs: (1/0/1)-(1023/254/63)d (1/0/1)-(14945/254/63)r > hex: 00 00 01 01 0F FE FF FF C1 3E 00 00 21 81 4F 0E > > Logical partition > type: 011(0x0B)(DOS or Windows 95 with 32 bit FAT) > size: 117232mb #s(240091362) s(16128-240107489) > chs: (1/1/1)-(1023/254/63)d (1/1/1)-(14945/254/63)r > hex: 00 01 01 01 0B FE FF FF 3F 00 00 00 E2 80 4F 0E > > (others are unused) > > Which leads me to think it's a geometry problem.. the CHS reported by gpart is 14946/255/63, while in messages i have: > Apr 9 10:27:01 localhost /kernel: da0: 117246MB (240121728 512 byte sectors: 64H 32S/T 51710C) > > Any idea?? > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Wed Apr 9 09:39:14 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 19C6B37B401; Wed, 9 Apr 2003 09:39:14 -0700 (PDT) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2491043FA3; Wed, 9 Apr 2003 09:39:13 -0700 (PDT) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 13BAF2ED40B; Wed, 9 Apr 2003 09:39:13 -0700 (PDT) Date: Wed, 9 Apr 2003 09:39:13 -0700 From: Alfred Perlstein To: fs@freebsd.org Message-ID: <20030409163912.GF30960@elvis.mu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i cc: mckusick@freebsd.org Subject: bug: snapshots and df space X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2003 16:39:14 -0000 After I lost power to a machine it came back up and I tried to log in as a user, problem is that I was at "100%" disk capacity so my login failed. After cleaning up several hundred megs of object files the 'df' reported that I had space, however allocations kept failing. I'm pretty sure this is because the deleted files were still held in the snapshot, which makes sense, however I don't think that df(1) should be reporting space that isn't just there yet. -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' From owner-freebsd-fs@FreeBSD.ORG Wed Apr 9 09:40:57 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0037637B401; Wed, 9 Apr 2003 09:40:56 -0700 (PDT) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8B76A43FAF; Wed, 9 Apr 2003 09:40:56 -0700 (PDT) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 7CC7F2ED3D6; Wed, 9 Apr 2003 09:40:56 -0700 (PDT) Date: Wed, 9 Apr 2003 09:40:56 -0700 From: Alfred Perlstein To: fs@freebsd.org Message-ID: <20030409164056.GG30960@elvis.mu.org> References: <20030409163912.GF30960@elvis.mu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030409163912.GF30960@elvis.mu.org> User-Agent: Mutt/1.4.1i cc: mckusick@freebsd.org Subject: Re: bug: snapshots and df space X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2003 16:40:57 -0000 I forgot: FreeBSD lappy.reserved 5.0-CURRENT FreeBSD 5.0-CURRENT #0: Sat Mar 22 16:21:24 PST 2003 bright@lappy.reserved:/usr/obj/usr/src/sys/lappy i386 thanks, -Alfred * Alfred Perlstein [030409 09:39] wrote: > After I lost power to a machine it came back up and I tried to log in > as a user, problem is that I was at "100%" disk capacity so my login > failed. > > After cleaning up several hundred megs of object files the 'df' > reported that I had space, however allocations kept failing. > > I'm pretty sure this is because the deleted files were still held > in the snapshot, which makes sense, however I don't think that df(1) > should be reporting space that isn't just there yet. From owner-freebsd-fs@FreeBSD.ORG Fri Apr 11 19:01:15 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DE75D37B401; Fri, 11 Apr 2003 19:01:15 -0700 (PDT) Received: from tel.fer.hr (zg03-188.dialin.iskon.hr [213.191.135.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id ABE7C43F75; Fri, 11 Apr 2003 19:01:12 -0700 (PDT) (envelope-from zec@tel.fer.hr) Received: from tel.fer.hr (marko-tp.katoda.net [192.168.202.105] (may be forged)) by tp755 (8.12.6/8.12.6) with ESMTP id h3C1ddFL000252; Sat, 12 Apr 2003 03:39:46 +0200 (CEST) (envelope-from zec@tel.fer.hr) Message-ID: <3E976EBD.C3E66EF8@tel.fer.hr> Date: Sat, 12 Apr 2003 03:41:17 +0200 From: Marko Zec X-Mailer: Mozilla 4.8 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Content-Type: multipart/mixed; boundary="------------6E421FB4C20DD72F6491B716" cc: mckusick@McKusick.COM Subject: PATCH: Forcible delaying of UFS (soft)updates X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2003 02:01:16 -0000 This is a multi-part message in MIME format. --------------6E421FB4C20DD72F6491B716 Content-Type: text/plain; charset=iso-8859-2 Content-Transfer-Encoding: 7bit Here's a patch against 4.8-RELEASE kernel that allows disk writes on softupdates-enabled filesystems to be delayed for (theoretically) arbitrarily long periods of time. The motivation for such updating policy is surprisingly not purely suicidal - it can allow disks on laptops to spin down immediately after I/O operations and stay idle for longer periods of time, thus saving considerable amount of battery power. The patch introduces a new sysctl tunable vfs.sync_extdelay which controls the delay duration in seconds. If the variable is set to 0, the standard UFS synching policy is restored. The tunable can be either modified by hand or controlled by APM daemon using the attached rc.syncdelay script. When enabled, the extended delaying policy introduces some additional changes: - fsync() no longer flushes the buffers to disk, but returns immediately instead; - invoking sync() causes flushing of softupdates buffers to follow immediately, which was not the case before; - if one of the mounted filesystems becomes low on free space, which can happen if lot of data is written to the FS but FS metadata buffers are not updated to disk, flushing of all softupdates buffers is scheduled automatically; - if an I/O operation (typically read request) on ATA disk is performed, which is likely to cause the disk to be spinned up, the pending buffers are immediately flushed to the disk, but only if they were pending longer than what would be the case with normal updating policy. As I'm virtually clueless in FS concepts and theory I'm not sure if the above model doesn't shake the foundations of UFS operation, therefore I'd appreciate for more knowledgeable people to comment on the patch. Nevertheless, my laptop runs without glitches for the last two weeks with the extra delaying enabled, while happily achieving 5-10% longer battery operated periods, depending on disk utilization patterns. Cheers, Marko --------------6E421FB4C20DD72F6491B716 Content-Type: text/plain; charset=iso-8859-2; name="syncdelay.diff" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="syncdelay.diff" --- /usr/src/sys.org/dev/ata/ata-disk.c Thu Jan 30 08:19:59 2003 +++ dev/ata/ata-disk.c Sat Apr 12 00:31:26 2003 @@ -294,6 +294,7 @@ adstrategy(struct buf *bp) struct ad_softc *adp = bp->b_dev->si_drv1; int s; + stratcalls++; if (adp->device->flags & ATA_D_DETACHING) { bp->b_error = ENXIO; bp->b_flags |= B_ERROR; --- /usr/src/sys.org/kern/vfs_subr.c Sun Oct 13 18:19:12 2002 +++ kern/vfs_subr.c Sat Apr 12 01:56:16 2003 @@ -116,6 +116,10 @@ SYSCTL_INT(_vfs, OID_AUTO, reassignbufme static int nameileafonly = 0; SYSCTL_INT(_vfs, OID_AUTO, nameileafonly, CTLFLAG_RW, &nameileafonly, 0, ""); +int stratcalls = 0; +int sync_extdelay = 0; +SYSCTL_INT(_vfs, OID_AUTO, sync_extdelay, CTLFLAG_RW, &sync_extdelay, 0, ""); + #ifdef ENABLE_VFS_IOOPT int vfs_ioopt = 0; SYSCTL_INT(_vfs, OID_AUTO, ioopt, CTLFLAG_RW, &vfs_ioopt, 0, ""); @@ -137,7 +141,7 @@ static vm_zone_t vnode_zone; * The workitem queue. */ #define SYNCER_MAXDELAY 32 -static int syncer_maxdelay = SYNCER_MAXDELAY; /* maximum delay time */ +int syncer_maxdelay = SYNCER_MAXDELAY; /* maximum delay time */ time_t syncdelay = 30; /* max time to delay syncing data */ time_t filedelay = 30; /* time to delay syncing files */ SYSCTL_INT(_kern, OID_AUTO, filedelay, CTLFLAG_RW, &filedelay, 0, ""); @@ -145,7 +149,7 @@ time_t dirdelay = 29; /* time to delay SYSCTL_INT(_kern, OID_AUTO, dirdelay, CTLFLAG_RW, &dirdelay, 0, ""); time_t metadelay = 28; /* time to delay syncing metadata */ SYSCTL_INT(_kern, OID_AUTO, metadelay, CTLFLAG_RW, &metadelay, 0, ""); -static int rushjob; /* number of slots to run ASAP */ +int rushjob; /* number of slots to run ASAP */ static int stat_rush_requests; /* number of times I/O speeded up */ SYSCTL_INT(_debug, OID_AUTO, rush_requests, CTLFLAG_RW, &stat_rush_requests, 0, ""); @@ -177,6 +181,7 @@ vntblinit() { desiredvnodes = maxproc + cnt.v_page_count / 4; + TUNABLE_INT_FETCH("kern.maxvnodes", &desiredvnodes); minvnodes = desiredvnodes / 4; simple_lock_init(&mntvnode_slock); simple_lock_init(&mntid_slock); @@ -1119,7 +1124,7 @@ sched_sync(void) { struct synclist *slp; struct vnode *vp; - long starttime; + time_t starttime; int s; struct proc *p = updateproc; @@ -1127,8 +1132,6 @@ sched_sync(void) SHUTDOWN_PRI_LAST); for (;;) { - kproc_suspend_loop(p); - starttime = time_second; /* @@ -1198,8 +1201,25 @@ sched_sync(void) * matter as we are just trying to generally pace the * filesystem activity. */ - if (time_second == starttime) + if (time_second != starttime) + continue; + + if (sync_extdelay >= syncer_maxdelay) + while (syncer_delayno == 0 && rushjob == 0 && + abs(time_second - starttime) < sync_extdelay) { + stratcalls = 0; + tsleep(&lbolt, PPAUSE, "syncer", 0); + kproc_suspend_loop(p); + if (stratcalls != 0 && syncer_maxdelay < + abs(time_second - starttime)) { + rushjob = syncer_maxdelay; + break; + } + } + else { tsleep(&lbolt, PPAUSE, "syncer", 0); + kproc_suspend_loop(p); + } } } --- /usr/src/sys.org/kern/vfs_syscalls.c Thu Jan 2 18:26:18 2003 +++ kern/vfs_syscalls.c Sat Apr 12 01:55:48 2003 @@ -563,6 +563,9 @@ sync(p, uap) register struct mount *mp, *nmp; int asyncflag; + /* Notify sched_sync() to try flushing syncer_workitem_pending[*] */ + rushjob += syncer_maxdelay; + simple_lock(&mountlist_slock); for (mp = TAILQ_FIRST(&mountlist); mp != NULL; mp = nmp) { if (vfs_busy(mp, LK_NOWAIT, &mountlist_slock, p)) { @@ -2627,6 +2630,10 @@ fsync(p, uap) struct file *fp; vm_object_t obj; int error; + + /* Just return if we are artificially delaying disk syncs */ + if (sync_extdelay) + return (0); if ((error = getvnode(p->p_fd, SCARG(uap, fd), &fp)) != 0) return (error); --- /usr/src/sys.org/ufs/ffs/ffs_alloc.c Fri Sep 21 21:15:21 2001 +++ ufs/ffs/ffs_alloc.c Sat Apr 12 00:06:20 2003 @@ -125,6 +125,10 @@ ffs_alloc(ip, lbn, bpref, size, cred, bn #endif /* DIAGNOSTIC */ if (size == fs->fs_bsize && fs->fs_cstotal.cs_nbfree == 0) goto nospace; + /* Speedup flushing of syncer_wokitem_pending[*] if low on freespace */ + if (rushjob == 0 && + freespace(fs, fs->fs_minfree + 2) - numfrags(fs, size) < 0) + rushjob = syncer_maxdelay; if (cred->cr_uid != 0 && freespace(fs, fs->fs_minfree) - numfrags(fs, size) < 0) goto nospace; @@ -195,6 +199,10 @@ ffs_realloccg(ip, lbprev, bpref, osize, if (cred == NOCRED) panic("ffs_realloccg: missing credential"); #endif /* DIAGNOSTIC */ + /* Speedup flushing of syncer_wokitem_pending[*] if low on freespace */ + if (rushjob == 0 && + freespace(fs, fs->fs_minfree + 2) - numfrags(fs, nsize - osize) < 0) + rushjob = syncer_maxdelay; if (cred->cr_uid != 0 && freespace(fs, fs->fs_minfree) - numfrags(fs, nsize - osize) < 0) goto nospace; --- /usr/src/sys.org/sys/buf.h Sat Jan 25 20:02:23 2003 +++ sys/buf.h Sat Apr 12 00:30:48 2003 @@ -478,6 +478,7 @@ extern char *buffers; /* The buffer con extern int bufpages; /* Number of memory pages in the buffer pool. */ extern struct buf *swbuf; /* Swap I/O buffer headers. */ extern int nswbuf; /* Number of swap I/O buffer headers. */ +extern int stratcalls; /* I/O ops since last buffer sync */ extern TAILQ_HEAD(swqueue, buf) bswlist; extern TAILQ_HEAD(bqueues, buf) bufqueues[BUFFER_QUEUES]; --- /usr/src/sys.org/sys/vnode.h Sun Dec 29 19:19:53 2002 +++ sys/vnode.h Sat Apr 12 00:06:20 2003 @@ -294,6 +294,9 @@ extern struct vm_zone *namei_zone; extern int prtactive; /* nonzero to call vprint() */ extern struct vattr va_null; /* predefined null vattr structure */ extern int vfs_ioopt; +extern int rushjob; +extern int syncer_maxdelay; +extern int sync_extdelay; /* * Macro/function to check for client cache inconsistency w.r.t. leasing. --------------6E421FB4C20DD72F6491B716 Content-Type: text/plain; charset=iso-8859-2; name="apmd.conf" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="apmd.conf" # apmd Configuration File # # $FreeBSD: src/etc/apmd.conf,v 1.2.2.1 2000/12/12 22:48:18 dannyboy Exp $ # apm_event POWERSTATECHANGE { exec "/etc/rc.syncdelay"; } apm_event SUSPENDREQ { exec "/etc/rc.suspend"; } apm_event USERSUSPENDREQ { exec "sync && sync && sync"; #exec "sleep 1"; exec "apm -z"; } apm_event NORMRESUME, STANDBYRESUME { exec "/etc/rc.resume"; exec "/etc/rc.syncdelay"; } # resume event configuration for serial mouse users by # reinitializing a moused(8) connected to a serial port. # #apm_event NORMRESUME { # exec "kill -HUP `cat /var/run/moused.pid`"; #} # suspend request event configuration for ATA HDD users: # execute standby instead of suspend. # #apm_event SUSPENDREQ { # reject; # exec "sync && sync && sync"; # exec "sleep 1"; # exec "apm -Z"; #} # Sample entries for battery state monitoring #apm_battery 5% discharging { # exec "logger -p user.emerg battery status critical!"; # exec "echo T250L8CE-GE-C >/dev/speaker"; #} #apm_battery 1% discharging { # exec "logger -p user.emerg battery low - emergency suspend"; # exec "echo T250L16B+BA+AG+GF+FED+DC+CC >/dev/speaker"; # exec "apm -z"; #} #apm_battery 99% charging { # exec "logger -p user.notice battery fully charged"; #} # apmd Configuration ends here --------------6E421FB4C20DD72F6491B716 Content-Type: text/plain; charset=iso-8859-2; name="rc.syncdelay" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="rc.syncdelay" #!/bin/sh # # Copyright (c) 2003 Marko Zec # #include /usr/share/examples/bsd-style-copyright # # # /etc/rc.syncdelay # # Adjust disk syncing policy and delay on battery powered systems. # Invoked automatically by apmd(8) when power state change or resume # events occur. # AC_DELAY=0 # no delayed syncing BAT_DELAY=600 # sync every 10 minutes if [ `apm -a` -eq 1 ]; then # AC powered mode sysctl vfs.sync_extdelay=$AC_DELAY else # Battery powered mode # Allow delayed syncing only if enough battery capacity is available if [ `apm -l` -gt 3 ]; then sysctl vfs.sync_extdelay=$BAT_DELAY else sysctl vfs.sync_extdelay=0 fi fi exit 0 --------------6E421FB4C20DD72F6491B716-- From owner-freebsd-fs@FreeBSD.ORG Fri Apr 11 20:33:08 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2D58037B401; Fri, 11 Apr 2003 20:33:08 -0700 (PDT) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id CAEBC43FD7; Fri, 11 Apr 2003 20:33:07 -0700 (PDT) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id B82A92ED410; Fri, 11 Apr 2003 20:33:07 -0700 (PDT) Date: Fri, 11 Apr 2003 20:33:07 -0700 From: Alfred Perlstein To: Marko Zec Message-ID: <20030412033307.GR30960@elvis.mu.org> References: <3E976EBD.C3E66EF8@tel.fer.hr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3E976EBD.C3E66EF8@tel.fer.hr> User-Agent: Mutt/1.4.1i cc: freebsd-fs@freebsd.org cc: mckusick@McKusick.COM cc: freebsd-stable@freebsd.org Subject: Re: PATCH: Forcible delaying of UFS (soft)updates X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2003 03:33:08 -0000 * Marko Zec [030411 19:01] wrote: > > When enabled, the extended delaying policy introduces some additional > changes: > > - fsync() no longer flushes the buffers to disk, but returns immediately > instead; This is really the only bad thing I can see here, what about introducing a slight delay and seeing if one can coalesce the writes? Is this part really needed? Making fsync() not work is a good way to make any sort of userland based transactional system break badly. otherwise, way cool! -Alfred From owner-freebsd-fs@FreeBSD.ORG Sat Apr 12 02:12:16 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DD2E037B401; Sat, 12 Apr 2003 02:12:16 -0700 (PDT) Received: from dire.bris.ac.uk (dire.bris.ac.uk [137.222.10.60]) by mx1.FreeBSD.org (Postfix) with ESMTP id DF03D43FB1; Sat, 12 Apr 2003 02:12:15 -0700 (PDT) (envelope-from Jan.Grant@bristol.ac.uk) Received: from mail.ilrt.bris.ac.uk by dire.bris.ac.uk with SMTP-PRIV with ESMTP; Sat, 12 Apr 2003 10:07:40 +0100 Received: from cmjg (helo=localhost) by mail.ilrt.bris.ac.uk with local-esmtp (Exim 3.16 #1) id 194Gwq-00030X-00; Sat, 12 Apr 2003 10:05:20 +0100 Date: Sat, 12 Apr 2003 10:05:20 +0100 (BST) From: Jan Grant X-X-Sender: cmjg@mail.ilrt.bris.ac.uk To: Marko Zec In-Reply-To: <3E976EBD.C3E66EF8@tel.fer.hr> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: Jan Grant cc: freebsd-fs@freebsd.org cc: mckusick cc: freebsd-stable Subject: Re: PATCH: Forcible delaying of UFS (soft)updates X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2003 09:12:17 -0000 On Sat, 12 Apr 2003, Marko Zec wrote: > When enabled, the extended delaying policy introduces some additional > changes: > > - fsync() no longer flushes the buffers to disk, but returns immediately > instead; This is bad; the rest looks very interesting. -- jan grant, ILRT, University of Bristol. http://www.ilrt.bris.ac.uk/ Tel +44(0)117 9287088 Fax +44 (0)117 9287112 http://ioctl.org/jan/ Rereleasing dolphins into the wild since 1998. From owner-freebsd-fs@FreeBSD.ORG Sat Apr 12 06:55:33 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9041137B401; Sat, 12 Apr 2003 06:55:33 -0700 (PDT) Received: from sentry.24cl.com (174.113.sn.ct.dsl.thebiz.net [216.238.113.174]) by mx1.FreeBSD.org (Postfix) with ESMTP id C0AC643F75; Sat, 12 Apr 2003 06:55:32 -0700 (PDT) (envelope-from zlists@mgm51.com) Received: from winbloat (unknown [10.0.1.10]) by sentry.24cl.com (Postfix) with ESMTP id C47BF2942D; Sat, 12 Apr 2003 09:55:31 -0400 (EDT) Message-ID: <200304120955310896.003EC3EE@sentry.24cl.com> In-Reply-To: <1050134860.7300.0.camel@rushlight.kf8nh.apk.net> References: <3E976EBD.C3E66EF8@tel.fer.hr> <20030412033307.GR30960@elvis.mu.org> <1050134860.7300.0.camel@rushlight.kf8nh.apk.net> X-Mailer: Calypso Version 3.20.01.01 (4) Date: Sat, 12 Apr 2003 09:55:31 -0400 From: "MikeM" To: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Content-Type: text/plain; charset="us-ascii" Subject: Re: PATCH: Forcible delaying of UFS (soft)updates X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2003 13:55:33 -0000 On 4/12/2003 at 4:07 AM Brandon S. Allbery wrote: |On Fri, 2003-04-11 at 23:33, Alfred Perlstein wrote: |> * Marko Zec [030411 19:01] wrote: |> > - fsync() no longer flushes the buffers to disk, but returns |immediately |> > instead; |> |> This is really the only bad thing I can see here, what about |> introducing a slight delay and seeing if one can coalesce the |> writes? Is this part really needed? Making fsync() not work |> is a good way to makeany sort of userland based transactional |> system break badly. | |If you're running that kind of thing you really don't want to be |using extended delays anyway, I'd think. ============= Perhaps a second parm for this patch: enable_suicidal_fsync with a default of NO From owner-freebsd-fs@FreeBSD.ORG Sat Apr 12 09:23:20 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E571737B401; Sat, 12 Apr 2003 09:23:20 -0700 (PDT) Received: from tel.fer.hr (zg05-232.dialin.iskon.hr [213.191.138.233]) by mx1.FreeBSD.org (Postfix) with ESMTP id 53EF743FBF; Sat, 12 Apr 2003 09:23:19 -0700 (PDT) (envelope-from zec@tel.fer.hr) Received: from tel.fer.hr ([192.168.202.105]) by tel.fer.hr (8.12.6/8.12.6) with ESMTP id h3CGLHaA000350; Sat, 12 Apr 2003 18:21:32 +0200 (CEST) (envelope-from zec@tel.fer.hr) Message-ID: <3E983D60.CFCB950A@tel.fer.hr> Date: Sat, 12 Apr 2003 18:22:56 +0200 From: Marko Zec X-Mailer: Mozilla 4.8 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Alfred Perlstein References: <3E976EBD.C3E66EF8@tel.fer.hr> <20030412033307.GR30960@elvis.mu.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: freebsd-fs@freebsd.org cc: mckusick@McKusick.COM cc: freebsd-stable@freebsd.org Subject: Re: PATCH: Forcible delaying of UFS (soft)updates X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2003 16:23:21 -0000 Alfred Perlstein wrote: > * Marko Zec [030411 19:01] wrote: > > > > When enabled, the extended delaying policy introduces some additional > > changes: > > > > - fsync() no longer flushes the buffers to disk, but returns immediately > > instead; > > This is really the only bad thing I can see here, what about introducing > a slight delay and seeing if one can coalesce the writes? Is this > part really needed? Making fsync() not work is a good way to make > any sort of userland based transactional system break badly. The motivation for hacking fsync() was in preventing some common tasks from spinning up the disk. One example is the vi editor, which if I'm not mistaking calls fsync() on the same moment as one starts modifying the file, not to mention on explicit :w. If the disk would start spinning every now and than, the whole patch would than become pointless... Nevertheless, the fact that the modified fsync() just returns without doing anything useful doesn't mean the data will be lost - it will simply be delayed until the next coalesced updating occurs. Marko From owner-freebsd-fs@FreeBSD.ORG Sat Apr 12 09:37:22 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A2AF737B401; Sat, 12 Apr 2003 09:37:22 -0700 (PDT) Received: from tel.fer.hr (zg05-232.dialin.iskon.hr [213.191.138.233]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1541D43FBD; Sat, 12 Apr 2003 09:37:21 -0700 (PDT) (envelope-from zec@tel.fer.hr) Received: from tel.fer.hr ([192.168.202.105]) by tel.fer.hr (8.12.6/8.12.6) with ESMTP id h3CGZXaA000355; Sat, 12 Apr 2003 18:35:36 +0200 (CEST) (envelope-from zec@tel.fer.hr) Message-ID: <3E9840B8.F00E018F@tel.fer.hr> Date: Sat, 12 Apr 2003 18:37:12 +0200 From: Marko Zec X-Mailer: Mozilla 4.8 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: freebsd-stable@FreeBSD.ORG, freebsd-fs@FreeBSD.ORG References: <200304121438.h3CEct41030991@lurza.secnetix.de> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re: PATCH: Forcible delaying of UFS (soft)updates X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2003 16:37:23 -0000 Oliver Fromme wrote: > Marko Zec wrote: > > Here's a patch against 4.8-RELEASE kernel that allows disk writes on > > softupdates-enabled filesystems to be delayed for (theoretically) > > arbitrarily long periods of time. The motivation for such updating > > policy is surprisingly not purely suicidal - it can allow disks on > > laptops to spin down immediately after I/O operations and stay idle for > > longer periods of time, thus saving considerable amount of battery > > power. > > It would be very cool if you could have different delay > settings per filesystem. That would enable you to have > a large delay on /tmp, a medium delay on /var, and the > standard delay (i.e. more safety) on everything else. That would certainly be cool, however I have no clue if and how such model could be implemented. But what I can safely assume is that it wouldn't be nearly as simple as the original patch. > > - fsync() no longer flushes the buffers to disk, but returns immediately > > instead; > > I see some issues with that. Better make that tunable > separately (and probably default to off). I agree that additional tunable for controlling fsync() behavior couldn't hurt, however as explained in previous note I see the fsync() as the most common initiator of disk spinnups, so a method for suppressing it must be made available, otherwise the whole patch wouldn't make much sense... > > - if one of the mounted filesystems becomes low on free space, which can > > happen if lot of data is written to the FS but FS metadata buffers are > > not updated to disk, flushing of all softupdates buffers is scheduled > > automatically; > > That's cool, too. I've been bitten several times by the > bogus "no space left on device", due to soft-updates > delaying the freeing of file data. > > I assume that buffered data is also flushed to disk when > the system runs low on RAM, right? (I'm not a VFS/VM > expert, so that might be a stupid question.) As far as I understand how softupdates work, the algorithm ensures the data is _always_ written to disk before the metadata, so that shouldn't be a problem. Cheers, Marko From owner-freebsd-fs@FreeBSD.ORG Sat Apr 12 09:58:15 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 44E9637B401; Sat, 12 Apr 2003 09:58:15 -0700 (PDT) Received: from OLYMPIC.AD.HartBrothers.Com (olympic.ad.hartbrothers.com [63.102.100.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9547F43FAF; Sat, 12 Apr 2003 09:58:14 -0700 (PDT) (envelope-from davehart@davehart.com) MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft Exchange V6.5.6803.0 Date: Sat, 12 Apr 2003 16:58:13 -0000 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: PATCH: Forcible delaying of UFS (soft)updates Thread-Index: AcMBD+R/0P6Gl+PfRL69GhtFtTaURAAAnvQA From: "Dave Hart" To: , cc: mckusick@McKusick.COM cc: Alfred Perlstein Subject: RE: PATCH: Forcible delaying of UFS (soft)updates X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2003 16:58:15 -0000 Marko Zec said: > Alfred Perlstein wrote: >=20 > > * Marko Zec [030411 19:01] wrote: > > > > > > When enabled, the extended delaying policy introduces=20 > > > some additional changes: > > > > > > - fsync() no longer flushes the buffers to disk, but=20 > > > returns immediately instead; [...] > > Making fsync() not work is a good way to make any sort > > of userland based transactional system break badly. [...] > If the disk would start spinning every now and than, > the whole patch would than become pointless... As I feared. > [...] the fact that the modified fsync() just returns=20 > without doing anything useful doesn't mean the data will be > lost - it will simply be delayed until the next coalesced > updating occurs. Unless, of course, your system or power happens to fail. Imagine you have a database program keeping track of banking transactions. This program uses fsync() to ensure its transaction logs are committed to reliable storage before indicating the transaction is completed. Suppose the moment after I withdraw $500 from an ATM, the operating system or hardware fails at the bank. With your change to fsync() to not commit to stable storage, I may have just won $500 courtesy of you. That is, the database software did all it could to ensure the $500 transaction was actually written to disk before authorizing the ATM to dispense cash, yet fsync() has decided it's not that important to do right away, so the transaction might well have not hit the disk before the catastrophe. For a perspective from the Windows world on the same sort of capability, check out the Win32 FlushFileBuffers spec: http://makeashorterlink.com/?E26B12F24 which is an alias for: http://msdn.microsoft.com/library/default.asp?url=3D/library/ en-us/fileio/base/flushfilebuffers.asp >From that page: "The FlushFileBuffers function writes all=20 of the buffered information for the specified file to disk." Such is the world of writing OS code -- optimizing for one situation may well break other important uses of the same code. Regards, Dave Hart davehart@davehart.com (who spent more time formatting text than writing, sigh) From owner-freebsd-fs@FreeBSD.ORG Sat Apr 12 10:06:45 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5207E37B401 for ; Sat, 12 Apr 2003 10:06:45 -0700 (PDT) Received: from mistral.imasy.or.jp (flets22-023.kamome.or.jp [218.45.22.23]) by mx1.FreeBSD.org (Postfix) with ESMTP id AD65F43FB1 for ; Sat, 12 Apr 2003 10:06:43 -0700 (PDT) (envelope-from mistral@imasy.or.jp) Received: from mistral.imasy.or.jp (localhost [IPv6:::1]) h3CH6eVl001532 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 13 Apr 2003 02:06:40 +0900 (JST) (envelope-from mistral@imasy.or.jp) Received: (from sarumaru@localhost) by mistral.imasy.or.jp (8.12.9/8.12.9/Submit) id h3CH6dAJ001531; Sun, 13 Apr 2003 02:06:39 +0900 (JST) (envelope-from sarumaru) From: mistral@imasy.or.jp (Yoshihiko Sarumaru) To: fs@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Mailer: mnews [version 1.22PL5] 2001-02/07(Wed) Date: Sun, 13 Apr 2003 02:06:39 +0900 Message-ID: <030413020639.M0101472@mistral.imasy.or.jp> cc: mistral@imasy.or.jp Subject: time stamp on msdosfs could not be set by general user X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2003 17:06:45 -0000 Hello, I've been wondering why cp -p files to msdosfs complains like this: mistral% cp -p somefile /dos/ cp: utimes: /dos/somefile: Operation not permitted cp: chmod: /dos/somefile: Operation not permitted I can understand errors about chmod, but I can not understand errors about utimes and modified time could not be set at all. This behavior is controllable by changing owner of mount point, but I feel this is unreasonable. Below patch ignores unmatching of user and file owner, and it imply this block will be always ignored. Comment out with #if 0 ~ #endif may better. Any objection ? --- sys/msdosfs/msdosfs_vnops.c.orig Wed Jan 1 23:38:45 2003 +++ sys/msdosfs/msdosfs_vnops.c Sun Apr 13 01:41:01 2003 @@ -498,9 +498,9 @@ } if (vap->va_atime.tv_sec != VNOVAL || vap->va_mtime.tv_sec != VNOVAL) { if (vp->v_mount->mnt_flag & MNT_RDONLY) return (EROFS); - if (cred->cr_uid != pmp->pm_uid && + if (/* cred->cr_uid != pmp->pm_uid */ 0 && (error = suser_xxx(cred, ap->a_p, PRISON_ROOT)) && ((vap->va_vaflags & VA_UTIMES_NULL) == 0 || (error = VOP_ACCESS(ap->a_vp, VWRITE, cred, ap->a_p)))) return (error); -- Yoshihiko Sarumaru mail: mistral@imasy.or.jp web: http://www.imasy.or.jp/~mistral/ From owner-freebsd-fs@FreeBSD.ORG Sat Apr 12 10:24:57 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D9EE537B401; Sat, 12 Apr 2003 10:24:57 -0700 (PDT) Received: from mail4.wi.rr.com (fe4.rdc-kc.rr.com [24.94.163.51]) by mx1.FreeBSD.org (Postfix) with ESMTP id E8A1143FAF; Sat, 12 Apr 2003 10:24:56 -0700 (PDT) (envelope-from hamilton@pobox.com) Received: from woodstock.nethamilton.net ([65.25.186.233]) by mail4.wi.rr.com with Microsoft SMTPSVC(5.5.1877.757.75); Sat, 12 Apr 2003 12:24:21 -0500 Received: by woodstock.nethamilton.net (Postfix, from userid 500) id 2432144EA; Sat, 12 Apr 2003 12:24:55 -0500 (CDT) Date: Sat, 12 Apr 2003 12:24:55 -0500 From: Jon Hamilton To: Dave Hart Message-ID: <20030412172455.GA85377@woodstock.nethamilton.net> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: cc: freebsd-fs@freebsd.org cc: freebsd-stable@freebsd.org Subject: Re: PATCH: Forcible delaying of UFS (soft)updates X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2003 17:24:58 -0000 Dave Hart , said on Sat Apr 12, 2003 [04:58:13 PM]: } Marko Zec said: [...] } > If the disk would start spinning every now and than, } > the whole patch would than become pointless... } } As I feared. } } > [...] the fact that the modified fsync() just returns } > without doing anything useful doesn't mean the data will be } > lost - it will simply be delayed until the next coalesced } > updating occurs. } } Unless, of course, your system or power happens to fail. } Imagine you have a database program keeping track of banking } transactions. This program uses fsync() to ensure its } transaction logs are committed to reliable storage before } indicating the transaction is completed. Suppose the moment } after I withdraw $500 from an ATM, the operating system or } hardware fails at the bank. Right. So in such a situation, the admin for that system would not enable this optional behavior. There probably aren't too many cases where mission critical financial transaction systems run on a laptop on which the desire is maximal battery life, which is the case from which this whole patch/discussion derives. It's a conscious tradeoff. -- Jon Hamilton hamilton@pobox.com From owner-freebsd-fs@FreeBSD.ORG Sat Apr 12 11:38:22 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 18D2237B401 for ; Sat, 12 Apr 2003 11:38:22 -0700 (PDT) Received: from laptop.tenebras.com (laptop.tenebras.com [66.92.188.18]) by mx1.FreeBSD.org (Postfix) with SMTP id 1EC6D43FA3 for ; Sat, 12 Apr 2003 11:38:19 -0700 (PDT) (envelope-from kudzu@tenebras.com) Received: (qmail 99638 invoked from network); 12 Apr 2003 18:38:18 -0000 Received: from sapphire.tenebras.com (HELO tenebras.com) (192.168.188.241) by 0 with SMTP; 12 Apr 2003 18:38:18 -0000 Message-ID: <3E985D19.1070206@tenebras.com> Date: Sat, 12 Apr 2003 11:38:17 -0700 From: Michael Sierchio User-Agent: Mozilla/5.0 (X11; U; Linux i386; en-US; rv:1.3) Gecko/20030312 X-Accept-Language: en-us, en, zh-cn, zh-tw MIME-Version: 1.0 To: Marko Zec References: <3E976EBD.C3E66EF8@tel.fer.hr> In-Reply-To: <3E976EBD.C3E66EF8@tel.fer.hr> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-fs@freebsd.org cc: mckusick@McKusick.COM cc: freebsd-stable@freebsd.org Subject: Re: PATCH: Forcible delaying of UFS (soft)updates X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2003 18:38:22 -0000 Marko Zec wrote: > - fsync() no longer flushes the buffers to disk, but returns immediately > instead; Any system that does this should be flushed down the toilet. Softupdates already breaks transaction semantics of FFS by breaking link()/unlink()/rename() etc. Don't make it worse. Many programs rely on fsync(). From owner-freebsd-fs@FreeBSD.ORG Sat Apr 12 11:54:05 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B8FE437B401 for ; Sat, 12 Apr 2003 11:54:05 -0700 (PDT) Received: from bilver.wjv.com (user38.net339.fl.sprint-hsd.net [65.40.24.38]) by mx1.FreeBSD.org (Postfix) with ESMTP id A540443FCB for ; Sat, 12 Apr 2003 11:54:03 -0700 (PDT) (envelope-from bv@wjv.com) Received: from bilver.wjv.com (localhost.wjv.com [127.0.0.1]) by bilver.wjv.com (8.12.9/8.12.9) with ESMTP id h3CIrmab061442 for ; Sat, 12 Apr 2003 14:53:49 -0400 (EDT) (envelope-from bv@wjv.com) Received: (from bv@localhost) by bilver.wjv.com (8.12.9/8.12.9/Submit) id h3CIrlQc061429 for freebsd-fs@freebsd.org; Sat, 12 Apr 2003 14:53:47 -0400 (EDT) Date: Sat, 12 Apr 2003 14:53:46 -0400 From: Bill Vermillion To: freebsd-fs@freebsd.org Message-ID: <20030412185346.GB52650@wjv.com> References: <20030412172455.GA85377@woodstock.nethamilton.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030412172455.GA85377@woodstock.nethamilton.net> Organization: W.J.Vermillion / Orlando - Winter Park ReplyTo: bv@wjv.com User-Agent: Mutt/1.5.1i X-Spam-Status: No, hits=-3.3 required=5.0 tests=IN_REP_TO,NIGERIAN_TRANSACTION_1,NOSPAM_INC, QUOTED_EMAIL_TEXT,REFERENCES,SPAM_PHRASE_01_02,USER_AGENT, USER_AGENT_MUTT version=2.43 Subject: Re: PATCH: Forcible delaying of UFS (soft)updates X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: bv@wjv.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2003 18:54:06 -0000 In the last exciting episode of the Jon Hamilton saga on Sat, Apr 12, 2003 at 12:24 , Jon Hamilton as heard to say: > Dave Hart , said on Sat Apr 12, 2003 [04:58:13 PM]: > } Marko Zec said: > [...] > } > If the disk would start spinning every now and than, > } > the whole patch would than become pointless... > } As I feared. > } > [...] the fact that the modified fsync() just returns > } > without doing anything useful doesn't mean the data will be > } > lost - it will simply be delayed until the next coalesced > } > updating occurs. > } Unless, of course, your system or power happens to fail. > } Imagine you have a database program keeping track of banking > } transactions. This program uses fsync() to ensure its > } transaction logs are committed to reliable storage before > } indicating the transaction is completed. Suppose the moment > } after I withdraw $500 from an ATM, the operating system or > } hardware fails at the bank. > Right. So in such a situation, the admin for that system would not > enable this optional behavior. There probably aren't too many cases > where mission critical financial transaction systems run on a laptop > on which the desire is maximal battery life, which is the case from > which this whole patch/discussion derives. It's a conscious tradeoff. I think 'the admin could enable this optional behaviour' is the wrong approach. I think it should be for laptops the admin could >disable< the feature. By default make everyting as robust and failsafe as possible. In the past enable options aren't always enabled when they should be. Default to secure and stable and let the admins change it. Bill -- Bill Vermillion - bv @ wjv . com From owner-freebsd-fs@FreeBSD.ORG Sat Apr 12 13:02:18 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 886E937B401; Sat, 12 Apr 2003 13:02:18 -0700 (PDT) Received: from salmon.maths.tcd.ie (salmon.maths.tcd.ie [134.226.81.11]) by mx1.FreeBSD.org (Postfix) with SMTP id 53CC343FB1; Sat, 12 Apr 2003 13:02:17 -0700 (PDT) (envelope-from iedowse@maths.tcd.ie) Received: from walton.maths.tcd.ie by salmon.maths.tcd.ie with SMTP id ; 12 Apr 2003 21:02:16 +0100 (BST) To: Marko Zec In-Reply-To: Your message of "Sat, 12 Apr 2003 03:41:17 +0200." <3E976EBD.C3E66EF8@tel.fer.hr> Date: Sat, 12 Apr 2003 21:02:13 +0100 From: Ian Dowse Message-ID: <200304122102.aa53041@salmon.maths.tcd.ie> cc: freebsd-fs@freebsd.org cc: mckusick@McKusick.COM cc: freebsd-stable@freebsd.org Subject: Re: PATCH: Forcible delaying of UFS (soft)updates X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2003 20:02:19 -0000 In message <3E976EBD.C3E66EF8@tel.fer.hr>, Marko Zec writes: >Here's a patch against 4.8-RELEASE kernel that allows disk writes on >softupdates-enabled filesystems to be delayed for (theoretically) >arbitrarily long periods of time. The motivation for such updating >policy is surprisingly not purely suicidal - it can allow disks on >laptops to spin down immediately after I/O operations and stay idle for >longer periods of time, thus saving considerable amount of battery >power. Looks interesting. A while ago I was reading the spec of some IBM ATA hard disk, and discovered that there is a "delayed write" feature built into most ATA disks that is extremely useful for keeping a laptop disk spun down. When the feature is enabled, the disk behaves normally until it spins down due to the standard ATA spindown timeout. Then it enters the delayed write mode, and all further writes to the disk go just to the disk cache and the disk is not spun up. Finally, when for any reason the disk needs to be spun up (cache is full, or a read of an uncached sector occurs), the cache is flushed as soon as the disk spins up. Assuming this is was happens (it's mostly based on observation rather than documentation), you get a much smaller window where the disk is potentially inconsistent, and automatic triggering of the writes only when they are necessary. Below is simple script I use to turn on the feature when running on battery power (using ACPI), and the -CURRENT patches that allow the spindown delay and delayed write features to be controlled with atacontrol (I mailed the patches to Soren a while ago). Ian #!/bin/sh oacline="" while :; do sleep 5 acline=`sysctl -n hw.acpi.acline` if [ "X$acline" = "X$oacline" ]; then continue fi oacline="$acline"; case "$acline" in 1) atacontrol standby 0 0 300 atacontrol delayed_write 0 0 0 ;; 0) atacontrol standby 0 0 20 atacontrol delayed_write 0 0 1 ;; esac done Index: sys/sys/ata.h =================================================================== RCS file: /dump/FreeBSD-CVS/src/sys/sys/ata.h,v retrieving revision 1.17 diff -u -r1.17 ata.h --- sys/sys/ata.h 22 Mar 2003 12:18:20 -0000 1.17 +++ sys/sys/ata.h 28 Mar 2003 02:42:27 -0000 @@ -370,6 +370,7 @@ #define ATARAIDSTATUS 11 #define ATAENCSTAT 12 #define ATAGMAXCHANNEL 13 +#define ATACMD 14 union { struct { @@ -409,6 +410,20 @@ int v05; int v12; } enclosure; + struct { + int flags; /* info about the request */ +#define ATA_CMD_CTRL 0x00 +#define ATA_CMD_READ 0x01 +#define ATA_CMD_WRITE 0x02 + + u_int8_t command; /* command code */ + u_int64_t lba; /* lba address */ + u_int16_t count; /* sector count */ + u_int8_t feature; /* feature modifier bits */ + + caddr_t databuf; /* I/O data buffer */ + int datalen; /* length of data buffer */ + } ata; struct { char ccb[16]; caddr_t data; Index: sys/dev/ata/ata-all.c =================================================================== RCS file: /dump/FreeBSD-CVS/src/sys/dev/ata/ata-all.c,v retrieving revision 1.175 diff -u -r1.175 ata-all.c --- sys/dev/ata/ata-all.c 30 Mar 2003 09:27:59 -0000 1.175 +++ sys/dev/ata/ata-all.c 1 Apr 2003 12:27:07 -0000 @@ -355,6 +355,28 @@ sizeof(struct ata_params)); return 0; + case ATACMD: { + struct ata_device *atadev; + + if (!device || !(ch = device_get_softc(device))) + return ENXIO; + if (!(atadev = &ch->device[iocmd->device]) || + !(ch->devices & (iocmd->device == MASTER ? + ATA_ATA_MASTER : ATA_ATA_SLAVE))) + return ENXIO; + if (iocmd->u.ata.flags != ATA_CMD_CTRL) + return EOPNOTSUPP; + + error = 0; + ATA_SLEEPLOCK_CH(ch, ATA_CONTROL); + if (ata_command(atadev, iocmd->u.ata.command, iocmd->u.ata.lba, + iocmd->u.ata.count, iocmd->u.ata.feature, + ATA_WAIT_INTR) != 0) + error = EIO; + ATA_UNLOCK_CH(ch); + return error; + } + case ATAENCSTAT: { struct ata_device *atadev; Index: sbin/atacontrol/atacontrol.8 =================================================================== RCS file: /dump/FreeBSD-CVS/src/sbin/atacontrol/atacontrol.8,v retrieving revision 1.22 diff -u -r1.22 atacontrol.8 --- sbin/atacontrol/atacontrol.8 23 Dec 2002 15:30:40 -0000 1.22 +++ sbin/atacontrol/atacontrol.8 31 Jan 2003 00:57:52 -0000 @@ -25,7 +25,7 @@ .\" .\" $FreeBSD: src/sbin/atacontrol/atacontrol.8,v 1.22 2002/12/23 15:30:40 ru Exp $ .\" -.Dd May 17, 2001 +.Dd August 18, 2002 .Dt ATACONTROL 8 .Os .Sh NAME @@ -72,6 +72,21 @@ .Ar channel device .Nm .Ic list +.Nm +.Ic idle +.Ar channel device +.Op seconds +.Nm +.Ic standby +.Ar channel device +.Op seconds +.Nm +.Ic sleep +.Ar channel device +.Nm +.Ic delayed_write +.Ar channel device +.Op 0 | 1 .Sh DESCRIPTION The .Nm @@ -208,6 +223,27 @@ Fan RPM speed, enclosure temperature, 5V and 12V levels are shown. .It Ic list Show info about all attached devices on all active controllers. +.It Ic idle +Set the idle timeout on the specified device. +If no timeout is given, put the device into the idle state immediately. +.It Ic standby +Set the standby timeout on the specified device. +If no timeout is given, put the device into the standby state immediately. +.It Ic sleep +Put the device into sleep mode. +Since this effectively powers down the device, settings configured by +the driver are lost, so this should not be used on an active drive. +Use +.Nm +.Ic reinit +to reinitialize the device for later use. +.It Ic delayed_write +Enable or disable the delayed write feature on the specified device. +When delayed writes are enabled on devices that support this feature, +writes that occur while the disk is spun down are stored in the +drive cache only. +Once the cache becomes full or the disk is spun up (e.g. for a read +operation), the cached writes are immediately flushed to the disk. .El .Sh EXAMPLES To see the devices' current access modes, use the command line: Index: sbin/atacontrol/atacontrol.c =================================================================== RCS file: /dump/FreeBSD-CVS/src/sbin/atacontrol/atacontrol.c,v retrieving revision 1.20 diff -u -r1.20 atacontrol.c --- sbin/atacontrol/atacontrol.c 22 Mar 2003 12:18:20 -0000 1.20 +++ sbin/atacontrol/atacontrol.c 1 Apr 2003 13:26:51 -0000 @@ -249,7 +249,7 @@ main(int argc, char **argv) { struct ata_cmd iocmd; - int fd, maxunit, unit; + int enable, fd, idle, maxunit, unit; if ((fd = open("/dev/ata", O_RDWR)) < 0) err(1, "control device not found"); @@ -427,6 +427,43 @@ mode2str(iocmd.u.mode.mode[0]), mode2str(iocmd.u.mode.mode[1])); } + } + else if ((!strcmp(argv[1], "idle") || !strcmp(argv[1], "standby") || + !strcmp(argv[1], "sleep")) && argc == 4) { + iocmd.cmd = ATACMD; + iocmd.device = atoi(argv[3]); + iocmd.u.ata.flags = ATA_CMD_CTRL; + iocmd.u.ata.command = !strcmp(argv[1], "idle") ? 0xe1 : + !strcmp(argv[1], "standby") ? 0xe0 : 0xe6; + if (ioctl(fd, IOCATA, &iocmd) < 0) + err(1, "ioctl(ATACMD)"); + } + else if ((!strcmp(argv[1], "idle") || !strcmp(argv[1], "standby")) && + argc == 5) { + idle = atoi(argv[4]); + if (idle > 19800) + errx(1, "Maximum idle time is 19800 seconds"); + if (idle <= 240*5) + iocmd.u.ata.count = (idle + 4) / 5; + else + iocmd.u.ata.count = idle / (30*60) + 240; + + iocmd.cmd = ATACMD; + iocmd.device = atoi(argv[3]); + iocmd.u.ata.flags = ATA_CMD_CTRL; + iocmd.u.ata.command = !strcmp(argv[1], "idle") ? 0xe3 : 0xe2; + if (ioctl(fd, IOCATA, &iocmd) < 0) + err(1, "ioctl(ATACMD)"); + } + else if (!strcmp(argv[1], "delayed_write") && argc == 5) { + enable = atoi(argv[4]); + iocmd.cmd = ATACMD; + iocmd.device = atoi(argv[3]); + iocmd.u.ata.feature = enable ? 0x07 : 0x87; + iocmd.u.ata.flags = ATA_CMD_CTRL; + iocmd.u.ata.command = 0xfa; + if (ioctl(fd, IOCATA, &iocmd) < 0) + err(1, "ioctl(ATACMD)"); } else usage(); From owner-freebsd-fs@FreeBSD.ORG Sat Apr 12 13:32:24 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D497137B401; Sat, 12 Apr 2003 13:32:24 -0700 (PDT) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3B00C43F85; Sat, 12 Apr 2003 13:32:24 -0700 (PDT) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 02D712ED3CF; Sat, 12 Apr 2003 13:32:24 -0700 (PDT) Date: Sat, 12 Apr 2003 13:32:23 -0700 From: Alfred Perlstein To: Michael Sierchio Message-ID: <20030412203223.GA18848@elvis.mu.org> References: <3E976EBD.C3E66EF8@tel.fer.hr> <3E985D19.1070206@tenebras.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3E985D19.1070206@tenebras.com> User-Agent: Mutt/1.4.1i cc: freebsd-fs@freebsd.org cc: mckusick@McKusick.COM cc: freebsd-stable@freebsd.org Subject: Re: PATCH: Forcible delaying of UFS (soft)updates X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2003 20:32:25 -0000 * Michael Sierchio [030412 11:38] wrote: > Marko Zec wrote: > > >- fsync() no longer flushes the buffers to disk, but returns immediately > >instead; > > Any system that does this should be flushed down the toilet. Softupdates > already breaks transaction semantics of FFS by breaking > link()/unlink()/rename() > etc. How does it do that? From owner-freebsd-fs@FreeBSD.ORG Sat Apr 12 16:02:14 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4663137B401; Sat, 12 Apr 2003 16:02:14 -0700 (PDT) Received: from chiark.greenend.org.uk (chiark.greenend.org.uk [193.201.200.170]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5891043F85; Sat, 12 Apr 2003 16:02:13 -0700 (PDT) (envelope-from fanf@chiark.greenend.org.uk) Received: from fanf by chiark.greenend.org.uk with local (Exim 3.35 #1) id 194U0f-0004Pn-00 (Debian); Sun, 13 Apr 2003 00:02:09 +0100 To: zec@tel.fer.hr From: Tony Finch In-Reply-To: <3E976EBD.C3E66EF8@tel.fer.hr> Message-Id: Sender: Tony Finch Date: Sun, 13 Apr 2003 00:02:09 +0100 cc: freebsd-fs@freebsd.org cc: freebsd-stable@freebsd.org Subject: Re: PATCH: Forcible delaying of UFS (soft)updates X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2003 23:02:14 -0000 Marko Zec wrote: > >Here's a patch against 4.8-RELEASE kernel that allows disk writes on >softupdates-enabled filesystems to be delayed for (theoretically) >arbitrarily long periods of time. The motivation for such updating >policy is surprisingly not purely suicidal - it can allow disks on >laptops to spin down immediately after I/O operations and stay idle for >longer periods of time, thus saving considerable amount of battery >power. I've used a much simpler patch for a number of years, which allows you to increase the 30s syncer interval. See http://dotat.at/prog/buildworld/patches.src/04.syncdelay.patch.disabled Tony. -- f.a.n.finch http://dotat.at/ LANDS END TO ST DAVIDS HEAD INCLUDING THE BRISTOL CHANNEL: EAST OR SOUTHEAST 4, INCREASING 7, LOCALLY GALE 8, EASING SOUTHEAST 4 OR 5 LATER. FAIR, THEN RAIN, WITH RISK OF MIST. GOOD, BECOMING MODERATE OR POOR. SLIGHT OR MODERATE, LOCALLY ROUGH LATER. From owner-freebsd-fs@FreeBSD.ORG Sat Apr 12 20:54:48 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F199837B401 for ; Sat, 12 Apr 2003 20:54:47 -0700 (PDT) Received: from bilver.wjv.com (user38.net339.fl.sprint-hsd.net [65.40.24.38]) by mx1.FreeBSD.org (Postfix) with ESMTP id A431343F75 for ; Sat, 12 Apr 2003 20:54:46 -0700 (PDT) (envelope-from bv@wjv.com) Received: from bilver.wjv.com (localhost.wjv.com [127.0.0.1]) by bilver.wjv.com (8.12.9/8.12.9) with ESMTP id h3D3sRab073568 for ; Sat, 12 Apr 2003 23:54:29 -0400 (EDT) (envelope-from bv@wjv.com) Received: (from bv@localhost) by bilver.wjv.com (8.12.9/8.12.9/Submit) id h3D3sRPY073550 for freebsd-fs@freebsd.org; Sat, 12 Apr 2003 23:54:27 -0400 (EDT) Date: Sat, 12 Apr 2003 23:54:25 -0400 From: Bill Vermillion To: freebsd-fs@freebsd.org Message-ID: <20030413035425.GA681@wjv.com> References: <20030412172455.GA85377@woodstock.nethamilton.net> <20030412185346.GB52650@wjv.com> <20030412231802.GB85377@woodstock.nethamilton.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030412231802.GB85377@woodstock.nethamilton.net> Organization: W.J.Vermillion / Orlando - Winter Park ReplyTo: bv@wjv.com User-Agent: Mutt/1.5.1i X-Spam-Status: No, hits=-3.3 required=5.0 tests=IN_REP_TO,NIGERIAN_TRANSACTION_1,NOSPAM_INC, QUOTED_EMAIL_TEXT,REFERENCES,SPAM_PHRASE_01_02,USER_AGENT, USER_AGENT_MUTT version=2.43 Subject: Re: PATCH: Forcible delaying of UFS (soft)updates X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: bv@wjv.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Apr 2003 03:54:48 -0000 Throwing caution to the wind and speaking without thinking about what was being said on Sat, Apr 12, 2003 at 18:18 , Jon Hamilton blurted this: > Bill Vermillion , said on Sat Apr 12, 2003 [02:53:46 PM]: > } In the last exciting episode of the Jon Hamilton saga > } on Sat, Apr 12, 2003 at 12:24 , Jon Hamilton as heard to say: > } > Dave Hart , said on Sat Apr 12, 2003 [04:58:13 PM]: > } > } Marko Zec said: > } > [...] > } > } > If the disk would start spinning every now and than, > } > } > the whole patch would than become pointless... > } > } > } As I feared. > } > } > [...] the fact that the modified fsync() just returns > } > } > without doing anything useful doesn't mean the data will be > } > } > lost - it will simply be delayed until the next coalesced > } > } > updating occurs. > } > } Unless, of course, your system or power happens to fail. > } > } Imagine you have a database program keeping track of banking > } > } transactions. This program uses fsync() to ensure its > } > } transaction logs are committed to reliable storage before > } > } indicating the transaction is completed. Suppose the moment > } > } after I withdraw $500 from an ATM, the operating system or > } > } hardware fails at the bank. > } > Right. So in such a situation, the admin for that system would not > } > enable this optional behavior. There probably aren't too many cases > } > where mission critical financial transaction systems run on a laptop > } > on which the desire is maximal battery life, which is the case from > } > which this whole patch/discussion derives. It's a conscious tradeoff. > } I think 'the admin could enable this optional behaviour' is the > } wrong approach. > } I think it should be for laptops the admin could >disable< the > } feature. By default make everyting as robust and failsafe as > } possible. > I agree, and that's what I said. Unfortunately, I wasn't overly clear > about it. The optional behavior would be the _enabling_ of the patch > behavior. I feel better about it already :-) Thanks for the clarification. Bill -- Bill Vermillion - bv @ wjv . com