Date: Wed, 15 May 2013 14:34:18 +0200 From: "Ronald Klop" <ronald-freebsd8@klop.yi.org> To: "Outback Dingo" <outbackdingo@gmail.com> Cc: freebsd-fs@freebsd.org Subject: Re: FreeBSD 9-RELEASE zpool Out of File Descriptors ?? Message-ID: <op.ww4k7gmr8527sy@ronaldradial.versatec.local> In-Reply-To: <CAKYr3zznxT7iL01ts1Cs9Eur-QkkK5mkKnb=qKPG-2UX%2BraqEw@mail.gmail.com> References: <CAKYr3zwer=P=GhW9WyxrMpLv4%2BAkxBfJkuZk99RTHtRW%2B-%2B24g@mail.gmail.com> <op.ww4kn4wx8527sy@ronaldradial.versatec.local> <CAKYr3zznxT7iL01ts1Cs9Eur-QkkK5mkKnb=qKPG-2UX%2BraqEw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 15 May 2013 14:27:10 +0200, Outback Dingo <outbackdingo@gmail.com> wrote: > On Wed, May 15, 2013 at 8:22 AM, Ronald Klop > <ronald-freebsd8@klop.yi.org>wrote: > >> On Wed, 15 May 2013 14:13:32 +0200, Outback Dingo >> <outbackdingo@gmail.com> >> wrote: >> >> So it seems a new deployment we just built with zfsonroot mirror and a >>> 48TB >>> master pool is already out of File Descriptors??? >>> >>> pool: master >>> state: ONLINE >>> status: The pool is formatted using a legacy on-disk format. The pool >>> can >>> still be used, but some features are unavailable. >>> action: Upgrade the pool using 'zpool upgrade'. Once this is done, the >>> pool will no longer be accessible on software that does not >>> support >>> feature >>> flags. >>> scan: none requested >>> config: >>> >>> NAME STATE READ WRITE CKSUM >>> master ONLINE 0 0 0 >>> raidz3-0 ONLINE 0 0 0 >>> multipath/SATA_LUN01 ONLINE 0 0 0 >>> multipath/SATA_LUN03 ONLINE 0 0 0 >>> multipath/SATA_LUN04 ONLINE 0 0 0 >>> multipath/SATA_LUN05 ONLINE 0 0 0 >>> multipath/SATA_LUN07 ONLINE 0 0 0 >>> multipath/SATA_LUN08 ONLINE 0 0 0 >>> multipath/SATA_LUN09 ONLINE 0 0 0 >>> multipath/SATA_LUN10 ONLINE 0 0 0 >>> raidz3-1 ONLINE 0 0 0 >>> multipath/SATA_LUN11 ONLINE 0 0 0 >>> multipath/SATA_LUN12 ONLINE 0 0 0 >>> multipath/SATA_LUN13 ONLINE 0 0 0 >>> multipath/SATA_LUN14 ONLINE 0 0 0 >>> multipath/SATA_LUN15 ONLINE 0 0 0 >>> multipath/SATA_LUN16 ONLINE 0 0 0 >>> multipath/SATA_LUN17 ONLINE 0 0 0 >>> multipath/SATA_LUN18 ONLINE 0 0 0 >>> raidz3-2 ONLINE 0 0 0 >>> multipath/SATA_LUN19 ONLINE 0 0 0 >>> multipath/SATA_LUN20 ONLINE 0 0 0 >>> multipath/SATA_LUN21 ONLINE 0 0 0 >>> multipath/SATA_LUN22 ONLINE 0 0 0 >>> multipath/SATA_LUN23 ONLINE 0 0 0 >>> multipath/SATA_LUN24 ONLINE 0 0 0 >>> multipath/SATA_LUN26 ONLINE 0 0 0 >>> multipath/SATA_LUN27 ONLINE 0 0 0 >>> raidz3-3 ONLINE 0 0 0 >>> multipath/SATA_LUN29 ONLINE 0 0 0 >>> multipath/SATA_LUN30 ONLINE 0 0 0 >>> multipath/SATA_LUN32 ONLINE 0 0 0 >>> multipath/SATA_LUN33 ONLINE 0 0 0 >>> multipath/SATA_LUN35 ONLINE 0 0 0 >>> multipath/SATA_LUN36 ONLINE 0 0 0 >>> multipath/SATA_LUN37 ONLINE 0 0 0 >>> multipath/SATA_LUN38 ONLINE 0 0 0 >>> logs >>> multipath/SATA_LUN06 ONLINE 0 0 0 >>> cache >>> multipath/SATA_LUN02 ONLINE 0 0 0 >>> >>> errors: No known data errors >>> pool: tank >>> state: ONLINE >>> status: The pool is formatted using a legacy on-disk format. The pool >>> can >>> still be used, but some features are unavailable. >>> action: Upgrade the pool using 'zpool upgrade'. Once this is done, the >>> pool will no longer be accessible on software that does not >>> support >>> feature >>> flags. >>> scan: none requested >>> config: >>> >>> NAME STATE READ WRITE CKSUM >>> tank ONLINE 0 0 0 >>> mirror-0 ONLINE 0 0 0 >>> da34p3 ONLINE 0 0 0 >>> da35p3 ONLINE 0 0 0 >>> >>> errors: No known data errors >>> >>> while compiling a kernel, buildworld worked and installed ok >>> >>> lex -t >>> /usr/src/sys/modules/aic7xxx/**aicasm/../../../dev/aic7xxx/** >>> aicasm/aicasm_scan.l >>> >>>> aicasm_scan.c >>>> >>> lex -t -Pmm >>> /usr/src/sys/modules/aic7xxx/**aicasm/../../../dev/aic7xxx/** >>> aicasm/aicasm_macro_scan.l >>> >>>> aicasm_macro_scan.c >>>> >>> rm -f .depend_aicasm >>> mkdep -f .depend_aicasm -a -I. >>> -I/usr/src/sys/modules/**aic7xxx/aicasm/../../../dev/**aic7xxx/aicasm >>> -std=gnu99 >>> /usr/src/sys/modules/aic7xxx/**aicasm/../../../dev/aic7xxx/** >>> aicasm/aicasm.c >>> /usr/src/sys/modules/aic7xxx/**aicasm/../../../dev/aic7xxx/** >>> aicasm/aicasm_symbol.c >>> aicasm_gram.c aicasm_macro_gram.c aicasm_scan.c aicasm_macro_scan.c >>> Out of file descriptors >>> *** [.depend_aicasm] Error code 2 >>> >>> Stop in /usr/src/sys/modules/aic7xxx/**aicasm. >>> *** [buildkernel] Error code 1 >>> >>> Stop in /usr/src. >>> *** [buildkernel] Error code 1 >>> >> >> >> Do you know what a file descriptor is? >> >> There is something about it here: http://www.freebsd.org/doc/en/** >> books/handbook/configtuning-**kernel-limits.html<http://www.freebsd.org/doc/en/books/handbook/configtuning-kernel-limits.html>(search >> for 'file descriptor') >> Or more info in here: https://www.google.nl/search?** >> q=freebsd+Out+of+file+**descriptors<https://www.google.nl/search?q=freebsd+Out+of+file+descriptors>; >> >> It mainly says that you have more files open than your system is >> configured to allow. This is often produced by a bug in a program which >> does not close some files properly. >> >> > Yes I do.... but this is a brand new zfsonroot with barely any data on it > and > > sysctl -a | grep kern.openfiles > kern.openfiles: 68 > root@:/master/builder # sysctl -a | grep kern.maxfiles > kern.maxfiles: 24600 > kern.maxfilesperproc: 11095 > > the openfiles and maxfiles seems to be plenty, but im out of > descriptors, i > used to see this back in the 4.x days when you could format > ufs with larger inodes, but zfs ??? really? Is this kern.openfiles when idle or during your buildkernel? Did you check 'ulimit -a'? You say installworld worked ok while you are just now doing buildkernel. Are your kernel and world out of sync? Ronald.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?op.ww4k7gmr8527sy>