Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 15 May 2013 08:46:32 -0400
From:      Outback Dingo <outbackdingo@gmail.com>
To:        Ronald Klop <ronald-freebsd8@klop.yi.org>
Cc:        freebsd-fs@freebsd.org
Subject:   Re: FreeBSD 9-RELEASE zpool Out of File Descriptors ??
Message-ID:  <CAKYr3zznHFWJTiM5B980s7-2JDU99Dk2svf_%2BaUhhMZngFkayA@mail.gmail.com>
In-Reply-To: <op.ww4k7gmr8527sy@ronaldradial.versatec.local>
References:  <CAKYr3zwer=P=GhW9WyxrMpLv4%2BAkxBfJkuZk99RTHtRW%2B-%2B24g@mail.gmail.com> <op.ww4kn4wx8527sy@ronaldradial.versatec.local> <CAKYr3zznxT7iL01ts1Cs9Eur-QkkK5mkKnb=qKPG-2UX%2BraqEw@mail.gmail.com> <op.ww4k7gmr8527sy@ronaldradial.versatec.local>

next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, May 15, 2013 at 8:34 AM, Ronald Klop <ronald-freebsd8@klop.yi.org>wrote:

> On Wed, 15 May 2013 14:27:10 +0200, Outback Dingo <outbackdingo@gmail.com>
> wrote:
>
>  On Wed, May 15, 2013 at 8:22 AM, Ronald Klop <ronald-freebsd8@klop.yi.org
>> >**wrote:
>>
>>  On Wed, 15 May 2013 14:13:32 +0200, Outback Dingo <
>>> outbackdingo@gmail.com>
>>> wrote:
>>>
>>>  So it seems a new deployment we just built with zfsonroot mirror and a
>>>
>>>> 48TB
>>>> master pool is already out of File Descriptors???
>>>>
>>>>   pool: master
>>>>  state: ONLINE
>>>> status: The pool is formatted using a legacy on-disk format.  The pool
>>>> can
>>>>         still be used, but some features are unavailable.
>>>> action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
>>>>         pool will no longer be accessible on software that does not
>>>> support
>>>> feature
>>>>         flags.
>>>>   scan: none requested
>>>> config:
>>>>
>>>>         NAME                      STATE     READ WRITE CKSUM
>>>>         master                    ONLINE       0     0     0
>>>>           raidz3-0                ONLINE       0     0     0
>>>>             multipath/SATA_LUN01  ONLINE       0     0     0
>>>>             multipath/SATA_LUN03  ONLINE       0     0     0
>>>>             multipath/SATA_LUN04  ONLINE       0     0     0
>>>>             multipath/SATA_LUN05  ONLINE       0     0     0
>>>>             multipath/SATA_LUN07  ONLINE       0     0     0
>>>>             multipath/SATA_LUN08  ONLINE       0     0     0
>>>>             multipath/SATA_LUN09  ONLINE       0     0     0
>>>>             multipath/SATA_LUN10  ONLINE       0     0     0
>>>>           raidz3-1                ONLINE       0     0     0
>>>>             multipath/SATA_LUN11  ONLINE       0     0     0
>>>>             multipath/SATA_LUN12  ONLINE       0     0     0
>>>>             multipath/SATA_LUN13  ONLINE       0     0     0
>>>>             multipath/SATA_LUN14  ONLINE       0     0     0
>>>>             multipath/SATA_LUN15  ONLINE       0     0     0
>>>>             multipath/SATA_LUN16  ONLINE       0     0     0
>>>>             multipath/SATA_LUN17  ONLINE       0     0     0
>>>>             multipath/SATA_LUN18  ONLINE       0     0     0
>>>>           raidz3-2                ONLINE       0     0     0
>>>>             multipath/SATA_LUN19  ONLINE       0     0     0
>>>>             multipath/SATA_LUN20  ONLINE       0     0     0
>>>>             multipath/SATA_LUN21  ONLINE       0     0     0
>>>>             multipath/SATA_LUN22  ONLINE       0     0     0
>>>>             multipath/SATA_LUN23  ONLINE       0     0     0
>>>>             multipath/SATA_LUN24  ONLINE       0     0     0
>>>>             multipath/SATA_LUN26  ONLINE       0     0     0
>>>>             multipath/SATA_LUN27  ONLINE       0     0     0
>>>>           raidz3-3                ONLINE       0     0     0
>>>>             multipath/SATA_LUN29  ONLINE       0     0     0
>>>>             multipath/SATA_LUN30  ONLINE       0     0     0
>>>>             multipath/SATA_LUN32  ONLINE       0     0     0
>>>>             multipath/SATA_LUN33  ONLINE       0     0     0
>>>>             multipath/SATA_LUN35  ONLINE       0     0     0
>>>>             multipath/SATA_LUN36  ONLINE       0     0     0
>>>>             multipath/SATA_LUN37  ONLINE       0     0     0
>>>>             multipath/SATA_LUN38  ONLINE       0     0     0
>>>>         logs
>>>>           multipath/SATA_LUN06    ONLINE       0     0     0
>>>>         cache
>>>>           multipath/SATA_LUN02    ONLINE       0     0     0
>>>>
>>>> errors: No known data errors
>>>>   pool: tank
>>>>  state: ONLINE
>>>> status: The pool is formatted using a legacy on-disk format.  The pool
>>>> can
>>>>         still be used, but some features are unavailable.
>>>> action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
>>>>         pool will no longer be accessible on software that does not
>>>> support
>>>> feature
>>>>         flags.
>>>>   scan: none requested
>>>> config:
>>>>
>>>>         NAME        STATE     READ WRITE CKSUM
>>>>         tank        ONLINE       0     0     0
>>>>           mirror-0  ONLINE       0     0     0
>>>>             da34p3  ONLINE       0     0     0
>>>>             da35p3  ONLINE       0     0     0
>>>>
>>>> errors: No known data errors
>>>>
>>>> while compiling a kernel, buildworld worked and installed ok
>>>>
>>>> lex -t
>>>> /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/****
>>>> aicasm/aicasm_scan.l
>>>>
>>>>  aicasm_scan.c
>>>>>
>>>>>  lex -t  -Pmm
>>>> /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/****
>>>>
>>>> aicasm/aicasm_macro_scan.l
>>>>
>>>>  aicasm_macro_scan.c
>>>>>
>>>>>  rm -f .depend_aicasm
>>>> mkdep -f .depend_aicasm -a    -I.
>>>> -I/usr/src/sys/modules/****aic7xxx/aicasm/../../../dev/****
>>>> aic7xxx/aicasm
>>>> -std=gnu99
>>>> /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/****
>>>> aicasm/aicasm.c
>>>> /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/****
>>>>
>>>> aicasm/aicasm_symbol.c
>>>> aicasm_gram.c aicasm_macro_gram.c aicasm_scan.c aicasm_macro_scan.c
>>>> Out of file descriptors
>>>> *** [.depend_aicasm] Error code 2
>>>>
>>>> Stop in /usr/src/sys/modules/aic7xxx/****aicasm.
>>>>
>>>> *** [buildkernel] Error code 1
>>>>
>>>> Stop in /usr/src.
>>>> *** [buildkernel] Error code 1
>>>>
>>>>
>>>
>>> Do you know what a file descriptor is?
>>>
>>> There is something about it here: http://www.freebsd.org/doc/en/****<http://www.freebsd.org/doc/en/**>;
>>> books/handbook/configtuning-****kernel-limits.html<http://www.**
>>> freebsd.org/doc/en/books/**handbook/configtuning-kernel-**limits.html<http://www.freebsd.org/doc/en/books/handbook/configtuning-kernel-limits.html>>(search
>>> for 'file descriptor')
>>> Or more info in here: https://www.google.nl/search?****<https://www.google.nl/search?**>;
>>> q=freebsd+Out+of+file+****descriptors<https://www.**
>>> google.nl/search?q=freebsd+**Out+of+file+descriptors<https://www.google.nl/search?q=freebsd+Out+of+file+descriptors>;
>>> >
>>>
>>>
>>> It mainly says that you have more files open than your system is
>>> configured to allow. This is often produced by a bug in a program which
>>> does not close some files properly.
>>>
>>>
>>>  Yes I do.... but this is a brand new zfsonroot with barely any data on
>> it
>> and
>>
>> sysctl -a | grep kern.openfiles
>> kern.openfiles: 68
>> root@:/master/builder # sysctl -a | grep kern.maxfiles
>> kern.maxfiles: 24600
>> kern.maxfilesperproc: 11095
>>
>> the openfiles and maxfiles seems to be plenty, but im out of descriptors,
>> i
>> used to see this back in the 4.x days when you could format
>> ufs with larger inodes, but zfs ??? really?
>>
>
>
> Is this kern.openfiles when idle or during your buildkernel?
> Did you check 'ulimit -a'?
> You say installworld worked ok while you are just now doing buildkernel.
> Are your kernel and world out of sync?
>
>
well, i was upgrading FreeBSD 9-RELEASE to CURRENT, did a make buildworld,
then a make installworld, when i went to do a make buildkernel
thats the error i got, so id say, yes it is "out of sync"

my ulimits are
ulimit -a
cpu time               (seconds, -t)  unlimited
file size           (512-blocks, -f)  unlimited
data seg size           (kbytes, -d)  33554432
stack size              (kbytes, -s)  524288
core file size      (512-blocks, -c)  unlimited
max memory size         (kbytes, -m)  unlimited
locked memory           (kbytes, -l)  unlimited
max user processes              (-u)  5547
open files                      (-n)  11095
virtual mem size        (kbytes, -v)  unlimited
swap limit              (kbytes, -w)  unlimited
sbsize                   (bytes, -b)  unlimited
pseudo-terminals                (-p)  unlimited



> Ronald.
>



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAKYr3zznHFWJTiM5B980s7-2JDU99Dk2svf_%2BaUhhMZngFkayA>