From owner-freebsd-fs@FreeBSD.ORG Wed May 15 16:03:44 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 6885DEC2 for ; Wed, 15 May 2013 16:03:44 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-wi0-x230.google.com (mail-wi0-x230.google.com [IPv6:2a00:1450:400c:c05::230]) by mx1.freebsd.org (Postfix) with ESMTP id DFEAD96C for ; Wed, 15 May 2013 16:03:43 +0000 (UTC) Received: by mail-wi0-f176.google.com with SMTP id hr14so5333937wib.15 for ; Wed, 15 May 2013 09:03:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:content-type:mime-version:subject:from:in-reply-to:date :cc:message-id:references:to:x-mailer:x-gm-message-state; bh=pR/V/NmcH/P5rNTgJoJbecwEFyU11iOkuQnw0gkPSPc=; b=ISbwf3KoPf6M1kdqX4WGtHBXA6KSIaEVGkh/rIAMiYKLndnn3FDA2NiLvGRSuTDp+M sj+mZ/nluZja/ubElAFBtJQwG4VagdbXvSlvCI2LWBdMv1bsgmuuRiySpcgLGoTwYCi0 lqpl56VCCQ5SAvdcBCTBHdrEESA0V57mOMPeB4HqtI/GK8GTLA9VkRqr5rMAKpT+e6mJ j6EUwDJgNdI+rN6frUn0pKK7L5Ls1ub2oI04AA3GzewhF9WRXydlX+8mciAme5DYVa5X OHH3TxizBEwAvPBcVk/EDrNysiKFyn1HzjSu2ehKmow0zeMEmOUNGqfur4RQTipJIRZO XkhQ== X-Received: by 10.194.161.166 with SMTP id xt6mr5350349wjb.28.1368633817974; Wed, 15 May 2013 09:03:37 -0700 (PDT) Received: from dfleuriot.paris.hi-media-techno.com ([83.167.62.196]) by mx.google.com with ESMTPSA id ay7sm21529838wib.9.2013.05.15.09.03.35 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 15 May 2013 09:03:36 -0700 (PDT) Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: FreeBSD 9-RELEASE zpool Out of File Descriptors ?? From: Fleuriot Damien In-Reply-To: Date: Wed, 15 May 2013 18:03:35 +0200 Message-Id: <1CB05302-7A5C-4245-A997-65A70C3DAFB8@my.gd> References: <1CE05B33-7639-4F7F-9FD6-78B27D31E186@my.gd> To: Outback Dingo X-Mailer: Apple Mail (2.1503) X-Gm-Message-State: ALoCoQk5EPEVUt0rN4/bS4vnw9PrBxZ+/ZQOuP4uMRFBcOy9Ep69+BiRJdDbyCX8Lll7t1NmXIx0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 May 2013 16:03:44 -0000 On May 15, 2013, at 5:56 PM, Outback Dingo = wrote: >=20 >=20 >=20 > On Wed, May 15, 2013 at 11:49 AM, Fleuriot Damien wrote: >=20 > On May 15, 2013, at 2:46 PM, Outback Dingo = wrote: >=20 > > On Wed, May 15, 2013 at 8:34 AM, Ronald Klop = wrote: > > > >> On Wed, 15 May 2013 14:27:10 +0200, Outback Dingo = > >> wrote: > >> > >> On Wed, May 15, 2013 at 8:22 AM, Ronald Klop = >>>> **wrote: > >>> > >>> On Wed, 15 May 2013 14:13:32 +0200, Outback Dingo < > >>>> outbackdingo@gmail.com> > >>>> wrote: > >>>> > >>>> So it seems a new deployment we just built with zfsonroot mirror = and a > >>>> > >>>>> 48TB > >>>>> master pool is already out of File Descriptors??? > >>>>> > >>>>> pool: master > >>>>> state: ONLINE > >>>>> status: The pool is formatted using a legacy on-disk format. = The pool > >>>>> can > >>>>> still be used, but some features are unavailable. > >>>>> action: Upgrade the pool using 'zpool upgrade'. Once this is = done, the > >>>>> pool will no longer be accessible on software that does = not > >>>>> support > >>>>> feature > >>>>> flags. > >>>>> scan: none requested > >>>>> config: > >>>>> > >>>>> NAME STATE READ WRITE CKSUM > >>>>> master ONLINE 0 0 0 > >>>>> raidz3-0 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN01 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN03 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN04 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN05 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN07 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN08 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN09 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN10 ONLINE 0 0 0 > >>>>> raidz3-1 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN11 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN12 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN13 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN14 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN15 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN16 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN17 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN18 ONLINE 0 0 0 > >>>>> raidz3-2 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN19 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN20 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN21 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN22 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN23 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN24 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN26 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN27 ONLINE 0 0 0 > >>>>> raidz3-3 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN29 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN30 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN32 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN33 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN35 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN36 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN37 ONLINE 0 0 0 > >>>>> multipath/SATA_LUN38 ONLINE 0 0 0 > >>>>> logs > >>>>> multipath/SATA_LUN06 ONLINE 0 0 0 > >>>>> cache > >>>>> multipath/SATA_LUN02 ONLINE 0 0 0 > >>>>> > >>>>> errors: No known data errors > >>>>> pool: tank > >>>>> state: ONLINE > >>>>> status: The pool is formatted using a legacy on-disk format. = The pool > >>>>> can > >>>>> still be used, but some features are unavailable. > >>>>> action: Upgrade the pool using 'zpool upgrade'. Once this is = done, the > >>>>> pool will no longer be accessible on software that does = not > >>>>> support > >>>>> feature > >>>>> flags. > >>>>> scan: none requested > >>>>> config: > >>>>> > >>>>> NAME STATE READ WRITE CKSUM > >>>>> tank ONLINE 0 0 0 > >>>>> mirror-0 ONLINE 0 0 0 > >>>>> da34p3 ONLINE 0 0 0 > >>>>> da35p3 ONLINE 0 0 0 > >>>>> > >>>>> errors: No known data errors > >>>>> > >>>>> while compiling a kernel, buildworld worked and installed ok > >>>>> > >>>>> lex -t > >>>>> = /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/**** > >>>>> aicasm/aicasm_scan.l > >>>>> > >>>>> aicasm_scan.c > >>>>>> > >>>>>> lex -t -Pmm > >>>>> = /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/**** > >>>>> > >>>>> aicasm/aicasm_macro_scan.l > >>>>> > >>>>> aicasm_macro_scan.c > >>>>>> > >>>>>> rm -f .depend_aicasm > >>>>> mkdep -f .depend_aicasm -a -I. > >>>>> -I/usr/src/sys/modules/****aic7xxx/aicasm/../../../dev/**** > >>>>> aic7xxx/aicasm > >>>>> -std=3Dgnu99 > >>>>> = /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/**** > >>>>> aicasm/aicasm.c > >>>>> = /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/**** > >>>>> > >>>>> aicasm/aicasm_symbol.c > >>>>> aicasm_gram.c aicasm_macro_gram.c aicasm_scan.c = aicasm_macro_scan.c > >>>>> Out of file descriptors > >>>>> *** [.depend_aicasm] Error code 2 > >>>>> > >>>>> Stop in /usr/src/sys/modules/aic7xxx/****aicasm. > >>>>> > >>>>> *** [buildkernel] Error code 1 > >>>>> > >>>>> Stop in /usr/src. > >>>>> *** [buildkernel] Error code 1 > >>>>> > >>>>> > >>>> > >>>> Do you know what a file descriptor is? > >>>> > >>>> There is something about it here: = http://www.freebsd.org/doc/en/**** > >>>> books/handbook/configtuning-****kernel-limits.html >>>> = freebsd.org/doc/en/books/**handbook/configtuning-kernel-**limits.html>= (search > >>>> for 'file descriptor') > >>>> Or more info in here: = https://www.google.nl/search?**** > >>>> q=3Dfreebsd+Out+of+file+****descriptors >>>> = google.nl/search?q=3Dfreebsd+**Out+of+file+descriptors > >>>>> > >>>> > >>>> > >>>> It mainly says that you have more files open than your system is > >>>> configured to allow. This is often produced by a bug in a program = which > >>>> does not close some files properly. > >>>> > >>>> > >>>> Yes I do.... but this is a brand new zfsonroot with barely any = data on > >>> it > >>> and > >>> > >>> sysctl -a | grep kern.openfiles > >>> kern.openfiles: 68 > >>> root@:/master/builder # sysctl -a | grep kern.maxfiles > >>> kern.maxfiles: 24600 > >>> kern.maxfilesperproc: 11095 > >>> > >>> the openfiles and maxfiles seems to be plenty, but im out of = descriptors, > >>> i > >>> used to see this back in the 4.x days when you could format > >>> ufs with larger inodes, but zfs ??? really? > >>> > >> > >> > >> Is this kern.openfiles when idle or during your buildkernel? > >> Did you check 'ulimit -a'? > >> You say installworld worked ok while you are just now doing = buildkernel. > >> Are your kernel and world out of sync? > >> > >> > > well, i was upgrading FreeBSD 9-RELEASE to CURRENT, did a make = buildworld, > > then a make installworld, when i went to do a make buildkernel > > thats the error i got, so id say, yes it is "out of sync" >=20 >=20 > Wow wow wow, hold on a sec here. >=20 > You're installing your new world before the kernel ? >=20 >=20 > I've had no trouble upgrading several boxes from 8-STABLE to = 10.0-CURRENT, following the regular procedure as described at: > http://www.freebsd.org/doc/handbook/makeworld.html >=20 >=20 > Note that the correct order is: > - buildworld (or kernel-toolchain if you only want to test the kernel = and not do the whole world) > - buildkernel > - installkernel > - reboot > - mergemaster -p (preferably in single user, I've always run it = multiuser w/o problems) > - installworld > - mergemaster > - reboot >=20 > You don't want to run the new world on the old kernel. >=20 > yeah i know better, seems when i went to install the old kernel the = build had failed, i can get through it from here >=20 > it was odd because the CURRENT kernel failed to build with a yyparse = error, until after i had completed a buildworld >=20 > the other node i did is fine.......... =20 >=20 Perhaps their sources weren't at the same revision level ? Anyway, you wanna reinstall a 9.x world from the ISO now, then update = sources and redo things in order ;) As a sidenote, if your root is on UFS, you can use nextboot to = failsafe-try your new kernel.=