Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 18 Jun 2008 23:47:11 -0500
From:      Paul Schmehl <pschmehl_lists@tx.rr.com>
To:        freebsd-questions@freebsd.org
Subject:   Re: /var full
Message-ID:  <DB82314EA03D57CB11849EA5@Macintosh.local>
In-Reply-To: <20080619035949.GB8205@shepherd>
References:  <EA09BDBE04BB7B81413DB590@Macintosh.local> <20080619035949.GB8205@shepherd>

next in thread | previous in thread | raw e-mail | index | archive | help
--On June 18, 2008 11:59:49 PM -0400 Sahil Tandon <sahil@tandon.net> wrote:

> Paul Schmehl <pschmehl_lists@tx.rr.com> wrote:
>
>> At 10PM (local time) this evening, a server started reporting that /var
>> was  full.  When I ssh'd in to the server to investigate, df said /var
>> was at 2%  full (5.1G) and dh reported the same (5.1G).
>> /var/log/dmesg.today is full  of messages listing multiple entries with
>> the same inode number followed by  one entry listing dd as the culprit.
>>
>> +pid 730 (mysqld), uid 88 inumber 7089166 on /var: filesystem full
>
> [...]
>
>> Was this some sort of temporary glitch?  Or something more ominous?
>> Why  would toor be running dd?  Is it some sort of file recovery
>> routine  triggered by filesystem full messages?
>
> This appears to be mysql-related:
>

I gathered that from the error messages.

> 	http://dev.mysql.com/doc/refman/5.0/en/temporary-files.html

Hmmm..tmpdir is not defined in /etc/my.cnf, so if I'm reading this right, 
mysqld *should* use /tmp for its temporary files.  This server has a /tmp 
partition that is 3.2GB, so that should be more than ample space *if* 
mysqld is really using it.  It appears that it may be using /var/tmp 
instead, which would be incorrect behavior *if* I'm reading their docs 
right.  But this /var partition is 300GB, so that's a really, really huge 
temporary file.

>
> Also, what is the output of 'df -i /var'?
>

# df -i /var/
Filesystem  1K-blocks    Used     Avail Capacity iused    ifree %iused 
Mounted on
/dev/da1s1d 283737842 5397568 255641248     2%   20350 36673664    0% 
/var

> See recent thread on FreeBSD Forums for context:
>
> 	http://www.freebsdforums.org/forums/printthread.php?t=58071

Thanks.  At least I know I'm not the only one to have run into this oddity.

I'm not that knowledgeable of inodes.  My understanding is they are 
destroyed once a file is no longer in use.  Is that correct?  Is there any 
sort of history kept of file system activity that would identify what 
filename was identified by the inumbers listed in dmesg.today?  Or is that 
vain hope?

This is a 6.2 RELEASE system.  (Looks like it's time to upgrade to 7.0 
STABLE.)

Paul Schmehl
If it isn't already obvious,
my opinions are my own and not
those of my employer.




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?DB82314EA03D57CB11849EA5>