Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 22 May 2009 12:19:25 -0400
From:      Steve Bertrand <steve@ibctech.ca>
To:        Luke Dean <LukeD@pobox.com>
Cc:        freebsd-questions@freebsd.org
Subject:   Re: How to recover disk space after "filesystem full"
Message-ID:  <4A16D08D.1000900@ibctech.ca>
In-Reply-To: <alpine.BSF.2.00.0905220853530.55458@border.lukas.is-a-geek.org>
References:  <alpine.BSF.2.00.0905220853530.55458@border.lukas.is-a-geek.org>

next in thread | previous in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format.

--------------ms000107080500040800070801
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Luke Dean wrote:
> 
> Yes, it sounds like a stupid question, but let me tell the story.
> 
> The log for my dhcp server filled up /var last night, which meant that
> dhcpd was also unable to hand out new leases, which meant that I had
> effectively been DOSed.  I'll have to look into changing my logging
> policies.
> 
> So, to correct the problem, I log into the router, removed the big
> log and several other files in /var to free up some space, and assumed
> this would correct the problem.
> 
> It did not.
> Several minutes after freeing up a lot of space on /var, I continued
> to get "filesystem full" messages and "df" continued to show the
> capacity at >100%.  I checked "df -i" for the inodes too.  That was
> fine.  I ran a quick fsck to see if that might shock the system into
> seeing all the space that I'd freed up, but no good.
> 
> I ended up rebooting the box.
> 
> Was there any other possible solution I could've tried?

You have to restart the service that was holding the log file(s) open.
The system does not release the space while an application is 'using'
the file, even after it's been deleted.

> Why wouldn't the free space immediately appear as free?

Because technically, the space is not freed. "lsof" will help identify
which process(es) are holding a particular file open, if you see that
disk space is not recovered as expected after deletion:

# pkg_add -r lsof

pearl# lsof | grep auth.log

syslogd     850     root   15w    VREG      0,127      75199  237484
/var/log/auth.log

Steve

--------------ms000107080500040800070801
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIII/zCC
AtowggJDoAMCAQICEEs5xg/J3t77QWJ4SatV1HcwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UE
BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT
I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA5MDUwNzIzMTYxMFoX
DTEwMDUwNzIzMTYxMFowQjEfMB0GA1UEAxMWVGhhd3RlIEZyZWVtYWlsIE1lbWJlcjEfMB0G
CSqGSIb3DQEJARYQc3RldmVAaWJjdGVjaC5jYTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC
AQoCggEBAJSTRAjP1RVa87/mnZn+PBTbENgyhhBJ4rWApmaNcthzRdk2DB/49KrXx3EQP60w
Lj4KU0DFkiGNVj9BnVxRAx/WDXKxGC3uGGEG6gjyWv8KFMWMsH9mL7y7uNow1HueT6pZUf9o
yY8Ewd+01QpGi7FfXOae7lGHhbEwnEJGwz08ytRfLmH0KtEzlZanZZhwDGX5s1kIHnyxdACh
3byXY6Z2bOrx0rcrQHCnHJppxddR60F7igjaMuBFstE51h9XTgXDNKJbglqTug5ghGihNuP6
VsBN7ue62y96UGIE22TvKEcAQ665vQGjHqZeSzZYy+hWNOa27pWFmhlqFjx0x8MCAwEAAaMt
MCswGwYDVR0RBBQwEoEQc3RldmVAaWJjdGVjaC5jYTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3
DQEBBQUAA4GBAMOmjxjp2Xzk6ZHLwTgFDzVhm98RjRT3UXotKjNIR7SgwfWF5wkJrx4I+dXu
ui5ztMEq4bTTRgJ344MqE6uZiZlg+tBIFHZGCJfKdzsX4QuV2jmw0sR5dMaYxG6tlDB0YUMv
gTqzV7ZDpiusTMOZe9pP1PdxFhOcIJXtMQDj5LhuMIIC2jCCAkOgAwIBAgIQSznGD8ne3vtB
YnhJq1XUdzANBgkqhkiG9w0BAQUFADBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3Rl
IENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVt
YWlsIElzc3VpbmcgQ0EwHhcNMDkwNTA3MjMxNjEwWhcNMTAwNTA3MjMxNjEwWjBCMR8wHQYD
VQQDExZUaGF3dGUgRnJlZW1haWwgTWVtYmVyMR8wHQYJKoZIhvcNAQkBFhBzdGV2ZUBpYmN0
ZWNoLmNhMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAlJNECM/VFVrzv+admf48
FNsQ2DKGEEnitYCmZo1y2HNF2TYMH/j0qtfHcRA/rTAuPgpTQMWSIY1WP0GdXFEDH9YNcrEY
Le4YYQbqCPJa/woUxYywf2YvvLu42jDUe55PqllR/2jJjwTB37TVCkaLsV9c5p7uUYeFsTCc
QkbDPTzK1F8uYfQq0TOVlqdlmHAMZfmzWQgefLF0AKHdvJdjpnZs6vHStytAcKccmmnF11Hr
QXuKCNoy4EWy0TnWH1dOBcM0oluCWpO6DmCEaKE24/pWwE3u57rbL3pQYgTbZO8oRwBDrrm9
AaMepl5LNljL6FY05rbulYWaGWoWPHTHwwIDAQABoy0wKzAbBgNVHREEFDASgRBzdGV2ZUBp
YmN0ZWNoLmNhMAwGA1UdEwEB/wQCMAAwDQYJKoZIhvcNAQEFBQADgYEAw6aPGOnZfOTpkcvB
OAUPNWGb3xGNFPdRei0qM0hHtKDB9YXnCQmvHgj51e66LnO0wSrhtNNGAnfjgyoTq5mJmWD6
0EgUdkYIl8p3OxfhC5XaObDSxHl0xpjEbq2UMHRhQy+BOrNXtkOmK6xMw5l72k/U93EWE5wg
le0xAOPkuG4wggM/MIICqKADAgECAgENMA0GCSqGSIb3DQEBBQUAMIHRMQswCQYDVQQGEwJa
QTEVMBMGA1UECBMMV2VzdGVybiBDYXBlMRIwEAYDVQQHEwlDYXBlIFRvd24xGjAYBgNVBAoT
EVRoYXd0ZSBDb25zdWx0aW5nMSgwJgYDVQQLEx9DZXJ0aWZpY2F0aW9uIFNlcnZpY2VzIERp
dmlzaW9uMSQwIgYDVQQDExtUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgQ0ExKzApBgkqhkiG
9w0BCQEWHHBlcnNvbmFsLWZyZWVtYWlsQHRoYXd0ZS5jb20wHhcNMDMwNzE3MDAwMDAwWhcN
MTMwNzE2MjM1OTU5WjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRp
bmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3Vp
bmcgQ0EwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMSmPFVzVftOucqZWh5owHUEcJ3f
6f+jHuy9zfVb8hp2vX8MOmHyv1HOAdTlUAow1wJjWiyJFXCO3cnwK4Vaqj9xVsuvPAsH5/Ef
kTYkKhPPK9Xzgnc9A74r/rsYPge/QIACZNenprufZdHFKlSFD0gEf6e20TxhBEAeZBlyYLf7
AgMBAAGjgZQwgZEwEgYDVR0TAQH/BAgwBgEB/wIBADBDBgNVHR8EPDA6MDigNqA0hjJodHRw
Oi8vY3JsLnRoYXd0ZS5jb20vVGhhd3RlUGVyc29uYWxGcmVlbWFpbENBLmNybDALBgNVHQ8E
BAMCAQYwKQYDVR0RBCIwIKQeMBwxGjAYBgNVBAMTEVByaXZhdGVMYWJlbDItMTM4MA0GCSqG
SIb3DQEBBQUAA4GBAEiM0VCD6gsuzA2jZqxnD3+vrL7CF6FDlpSdf0whuPg2H6otnzYvwPQc
UCCTcDz9reFhYsPZOhl+hLGZGwDFGguCdJ4lUJRix9sncVcljd2pnDmOjCBPZV+V2vf3h9bG
CE6u9uo05RAaWzVNd+NWIXiC3CEZNd4ksdMdRv9dX2VPMYIDZDCCA2ACAQEwdjBiMQswCQYD
VQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UE
AxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEEs5xg/J3t77QWJ4SatV
1HcwCQYFKw4DAhoFAKCCAcMwGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0B
CQUxDxcNMDkwNTIyMTYxOTI1WjAjBgkqhkiG9w0BCQQxFgQUV6KXpdIyyrsbqxn0j3fzMm5k
RMcwUgYJKoZIhvcNAQkPMUUwQzAKBggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZI
hvcNAwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgwgYUGCSsGAQQBgjcQBDF4MHYwYjEL
MAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAq
BgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhBLOcYPyd7e+0Fi
eEmrVdR3MIGHBgsqhkiG9w0BCRACCzF4oHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRo
YXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBG
cmVlbWFpbCBJc3N1aW5nIENBAhBLOcYPyd7e+0FieEmrVdR3MA0GCSqGSIb3DQEBAQUABIIB
AEdo2w5nfeFYfmgo9hvBfQVGqUDHS5ZivxqUBcX7xvfRLq1gbrU0BWrp8/vJl590N9GQe91l
bMNmAeUWn3kLbfyyhnupEcHsjv9eivFWbievtI8U3KvEfS2juH0zSfl8zX9OLl4WEzywQZs9
7FjtB8ZHEssybRUgy5t3v2LAAeHJpcrTYUO4Zx3p4rJHVAvibJOI6772BwAdVkyYFrByiPNe
OwUfnQSDjkBYkqqTuaM6vS2IjWRokfu3arIQaze1acu84dVOJHg2aivc1JU9ooYYMSXj4/YQ
BD2qKMshjma/ff0huuRqjTRBlXMtbqWDeLDhNm6hD+PnMtI1HI+I87sAAAAAAAA=
--------------ms000107080500040800070801--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4A16D08D.1000900>