Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 11 Oct 2009 06:39:08 +0200
From:      Miroslav Lachman <000.fbsd@quip.cz>
To:        freebsd-fs@freebsd.org
Subject:   ZFS vs. df -h completely different size of filesystem
Message-ID:  <4AD1616C.8060504@quip.cz>

next in thread | raw e-mail | index | archive | help
Hi,
I am using ZFS on this machine for one year without any problems. The 
main purpose is to run jails on filesystems with ZFS quotas.

The biggest jail has 360G (322GB used) running Lighttpd. I must change 
IP for this jail, so I stop the jail, asign new IP alias on interface, 
change IP in config and start the jail again. But now the jail reports 
only 24G quota and by another accident I deleted stored files by rsync.

The strange thing is this:

root@cage ~/# df -h /vol0/jail/kurt2
Filesystem              Size    Used   Avail Capacity  Mounted on
tank/vol0/jail/kurt2     24G     19G    4.9G    80%    /vol0/jail/kurt2


root@cage ~/# zfs list | grep tank/vol0/jail/kurt2
tank/vol0/jail/kurt2                             355G  4.89G  19.2G 
/vol0/jail/kurt2
tank/vol0/jail/kurt2@manual-2009-08-18_11-04    7.29G      -   299G  -
tank/vol0/jail/kurt2@manual-2009-09-04_10-24    2.24G      -   308G  -
tank/vol0/jail/kurt2@manual-2009-09-28_11-41    21.4G      -   317G  -


root@cage ~/# zfs get -r quota tank/vol0/jail/kurt2
NAME                     PROPERTY  VALUE    SOURCE
tank/vol0/jail/kurt2     quota     360G     local

As you can see, df reports 24G size and zfs list shows 355G.

What is wrong? I tired to write some date on to this filesystem and it 
stoped after 24G were filled.

Here are more details:

root@cage ~/# zfs get all tank/vol0/jail/kurt2
NAME                  PROPERTY       VALUE                  SOURCE
tank/vol0/jail/kurt2  type           filesystem             -
tank/vol0/jail/kurt2  creation       Sat Feb 28  0:52 2009  -
tank/vol0/jail/kurt2  used           355G                   -
tank/vol0/jail/kurt2  available      4.89G                  -
tank/vol0/jail/kurt2  referenced     19.2G                  -
tank/vol0/jail/kurt2  compressratio  1.02x                  -
tank/vol0/jail/kurt2  mounted        yes                    -
tank/vol0/jail/kurt2  quota          360G                   local
tank/vol0/jail/kurt2  reservation    none                   default
tank/vol0/jail/kurt2  recordsize     128K                   default
tank/vol0/jail/kurt2  mountpoint     /vol0/jail/kurt2       inherited 
from tank/vol0
tank/vol0/jail/kurt2  sharenfs       off                    default
tank/vol0/jail/kurt2  checksum       on                     default
tank/vol0/jail/kurt2  compression    gzip                   inherited 
from tank/vol0
tank/vol0/jail/kurt2  atime          off                    inherited 
from tank
tank/vol0/jail/kurt2  devices        on                     default
tank/vol0/jail/kurt2  exec           on                     inherited 
from tank/vol0/jail
tank/vol0/jail/kurt2  setuid         on                     inherited 
from tank/vol0/jail
tank/vol0/jail/kurt2  readonly       off                    default
tank/vol0/jail/kurt2  jailed         off                    default
tank/vol0/jail/kurt2  snapdir        hidden                 default
tank/vol0/jail/kurt2  aclmode        groupmask              default
tank/vol0/jail/kurt2  aclinherit     secure                 default
tank/vol0/jail/kurt2  canmount       on                     default
tank/vol0/jail/kurt2  shareiscsi     off                    default
tank/vol0/jail/kurt2  xattr          off                    temporary
tank/vol0/jail/kurt2  copies         1                      default

root@cage ~/# df -ht zfs
Filesystem                    Size    Used   Avail Capacity  Mounted on
tank                           76G     16G     59G    22%    /tank
tank/vol0                      59G    128K     59G     0%    /vol0
tank/vol0/jail                 60G    605M     59G     1%    /vol0/jail
tank/vol0/jail/china           20G    786M     19G     4% 
/vol0/jail/china
tank/vol0/jail/costa           40G    527M     39G     1% 
/vol0/jail/costa
tank/vol0/jail/kurt2           24G     19G    4.9G    80% 
/vol0/jail/kurt2
tank/vol0/jail/tester1         20G    874M     19G     4% 
/vol0/jail/tester1
tank/vol0/jail/tvujweb         25G    494M     24G     2% 
/vol0/jail/tvujweb
tank/vol0/jail/yamaha          19G    1.2G     18G     6% 
/vol0/jail/yamaha
tank/vol0/jail/kurt2_clone    376G    317G     59G    84% 
/vol0/jail/kurt2_clone

root@cage ~/# zfs list -t filesystem
NAME                         USED  AVAIL  REFER  MOUNTPOINT
tank                         378G  59.2G  16.4G  /tank
tank/vol0                    361G  59.2G    19K  /vol0
tank/vol0/jail               361G  59.2G   605M  /vol0/jail
tank/vol0/jail/china         996M  19.0G   786M  /vol0/jail/china
tank/vol0/jail/costa         938M  39.1G   527M  /vol0/jail/costa
tank/vol0/jail/kurt2         355G  4.89G  19.2G  /vol0/jail/kurt2
tank/vol0/jail/kurt2_clone      0  59.2G   317G  /vol0/jail/kurt2_clone
tank/vol0/jail/tester1      1.23G  18.8G   874M  /vol0/jail/tester1
tank/vol0/jail/tvujweb       691M  24.3G   494M  /vol0/jail/tvujweb
tank/vol0/jail/yamaha       1.72G  18.3G  1.18G  /vol0/jail/yamaha


kurt2_clone was created as clone of snapshot 
kurt2@manual-2009-09-28_11-41 after this crazyness

Did somebody seen this error behavior?

I can get all the data from master server by rsync, but 320GB of data, 
it will take many hours. Is there any way to move the data from 
kurt2_clone to kurt2? (after I find a reason, why kurt2 is so small now)

I cannot use promote:
root@cage ~/# zfs promote tank/vol0/jail/kurt2_clone
cannot promote 'tank/vol0/jail/kurt2_clone': out of space

system: FreeBSD 7.2-RELEASE #1: Tue May  5 16:17:50 CEST 2009

root@cage ~/# zdb
tank
     version=6
     name='tank'
     state=0
     txg=4
     pool_guid=15882109565948440736
     hostid=2083894993
     hostname='cage.xxxxx.yyy'
     vdev_tree
         type='root'
         id=0
         guid=15882109565948440736
         children[0]
                 type='mirror'
                 id=0
                 guid=15485397551239445092
                 metaslab_array=14
                 metaslab_shift=32
                 ashift=9
                 asize=478648991744
                 children[0]
                         type='disk'
                         id=0
                         guid=7324230677133648509
                         path='/dev/ad4s2'
                         devid='ad:GEB531RE0B213Fs1'
                         whole_disk=0
                 children[1]
                         type='disk'
                         id=1
                         guid=16876684646990874563
                         path='/dev/ad6s2'
                         devid='ad:GEB531RE0BJ72Fs1'
                         whole_disk=0


Miroslav Lachman



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4AD1616C.8060504>