Date: Fri, 2 May 2003 14:13:46 +0200 (CEST) From: Michael van Elst <mlelstv@dev.de.cw.net> To: FreeBSD-gnats-submit@FreeBSD.org Subject: kern/51685: Unbounded inode allocation causes kernel to lock up Message-ID: <20030502121505.026FD8D7F5@sv1.dev.de.cw.net> Resent-Message-ID: <200305021220.h42CKA3Q007466@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 51685 >Category: kern >Synopsis: Unbounded inode allocation causes kernel to lock up >Confidential: no >Severity: serious >Priority: high >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Fri May 02 05:20:09 PDT 2003 >Closed-Date: >Last-Modified: >Originator: Michael van Elst >Release: FreeBSD 4.8-RELEASE i386 >Organization: C&W >Environment: System: FreeBSD pg6.dev.de.cw.net 4.8-RELEASE FreeBSD 4.8-RELEASE #0: Thu Apr 3 10:53:38 GMT 2003 root@freebsd-stable.sentex.ca:/usr/obj/usr/src/sys/GENERIC i386 >Description: For every cached filesystem object a 'vnode' and an associated 'inode' are allocated in memory. The inodes are allocated in the 'FFS node' memory region. When a vnode is no longer used it is either explicitely released to a free list or reclaimed by a kernel thread. The reclaim operation in vnlru_proc() also releases the associated filesystem data. The release operation done by calling vput() or vrele() on the other hand leave the associated filesystem data intact. It is released only when a new vnode needs to be allocated by recycling a vnode from the free list. The 'FFS node' region is limited to about 100MB, when it is filled up the kernel sleeps until some process has freed memory in that region. When no other process is allocating new vnodes this can only happen by vnlru_proc() pushing vnodes to the free list. vnlru_proc() however only runs when the number of vnodes that are not on the free list exceeds the system parameter 'kern.maxvnodes'. When the inode memory allocated for the remaining 'kern.maxvnodes' vnodes and for all vnodes on the free list that are still associated with filesystem data reaches the 100MB limit the system freezes. The vfs.vmiodirenable setting only amplifies the problem. Without directory caching you need to access more files to reach the limit. FreeBSD5 has the same issue, however the critical memory region is now handled by UMA and has no artificial limit. It is still limited to the size of a UMA zone. >How-To-Repeat: - run a FreeBSD4 system with vfs.vmiodirenable=1 - on a FFS filesystem create a directory with about one million files or subdirectories. - do a 'find -ls' over this directory or wait for some periodic task that does similar (e.g. weekly/310.locate). - watch the system slow down and lock up >Fix: No fix known, but some ideas: If vrele() and vput() would simply drop a vnode it could always be reclaimed by vnlru_proc(). Caching would be done only in the vnode layer but not also in the ffs layer. As an alternative vlnru_proc() could use a heuristic that prevents a resource shortage in the ffs layer, something like 'desiredusedvnodes' that is checked against the number of vnodes that carry filesystem specific data. >Release-Note: >Audit-Trail: >Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030502121505.026FD8D7F5>