From owner-freebsd-fs@FreeBSD.ORG Wed May 15 16:05:59 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 18E79F86 for ; Wed, 15 May 2013 16:05:59 +0000 (UTC) (envelope-from outbackdingo@gmail.com) Received: from mail-oa0-f42.google.com (mail-oa0-f42.google.com [209.85.219.42]) by mx1.freebsd.org (Postfix) with ESMTP id D90D1993 for ; Wed, 15 May 2013 16:05:58 +0000 (UTC) Received: by mail-oa0-f42.google.com with SMTP id i10so2442287oag.29 for ; Wed, 15 May 2013 09:05:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=K93EynZg2cPMBj8OqTz7BCG//MlLno5Yhzpxm02bYNc=; b=BE4IeGLeOUoZvzIROda+Mk5WEbqTktF6mY8nwyhe08flmJooFUTcrCcqh7dvUsua7R OGIxrS+9JyqU677MNuxFGY/DG1EWual0jmqINBNqrGifHcZQZzGZsPlkv1YX4KO4vPFd v08J3jdzTkhgroydVIce2uQHdZnMqnNiVwNzRpF/UgPLAm8xPyCMURNNQ7jKBgoCxtex D5MROUcTdMkfHzz4uD+Rd6HQtZAHbCwCUZk7i1WEZsVDXWO+LLPkv0Oyyv4zq7PVj29/ hPRdRBzRjK3xY4hJyXe3HakAEWhCjm5ctjnwtgZiZXRphEBZswPtXTdmoGiAn7OzvbS8 mPRg== MIME-Version: 1.0 X-Received: by 10.60.93.67 with SMTP id cs3mr19147191oeb.88.1368633957957; Wed, 15 May 2013 09:05:57 -0700 (PDT) Received: by 10.76.96.49 with HTTP; Wed, 15 May 2013 09:05:57 -0700 (PDT) In-Reply-To: <1CB05302-7A5C-4245-A997-65A70C3DAFB8@my.gd> References: <1CE05B33-7639-4F7F-9FD6-78B27D31E186@my.gd> <1CB05302-7A5C-4245-A997-65A70C3DAFB8@my.gd> Date: Wed, 15 May 2013 12:05:57 -0400 Message-ID: Subject: Re: FreeBSD 9-RELEASE zpool Out of File Descriptors ?? From: Outback Dingo To: Fleuriot Damien Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 May 2013 16:05:59 -0000 On Wed, May 15, 2013 at 12:03 PM, Fleuriot Damien wrote: > > On May 15, 2013, at 5:56 PM, Outback Dingo wrote: > > > > > On Wed, May 15, 2013 at 11:49 AM, Fleuriot Damien wrote: > >> >> On May 15, 2013, at 2:46 PM, Outback Dingo >> wrote: >> >> > On Wed, May 15, 2013 at 8:34 AM, Ronald Klop < >> ronald-freebsd8@klop.yi.org>wrote: >> > >> >> On Wed, 15 May 2013 14:27:10 +0200, Outback Dingo < >> outbackdingo@gmail.com> >> >> wrote: >> >> >> >> On Wed, May 15, 2013 at 8:22 AM, Ronald Klop < >> ronald-freebsd8@klop.yi.org >> >>>> **wrote: >> >>> >> >>> On Wed, 15 May 2013 14:13:32 +0200, Outback Dingo < >> >>>> outbackdingo@gmail.com> >> >>>> wrote: >> >>>> >> >>>> So it seems a new deployment we just built with zfsonroot mirror and >> a >> >>>> >> >>>>> 48TB >> >>>>> master pool is already out of File Descriptors??? >> >>>>> >> >>>>> pool: master >> >>>>> state: ONLINE >> >>>>> status: The pool is formatted using a legacy on-disk format. The >> pool >> >>>>> can >> >>>>> still be used, but some features are unavailable. >> >>>>> action: Upgrade the pool using 'zpool upgrade'. Once this is done, >> the >> >>>>> pool will no longer be accessible on software that does not >> >>>>> support >> >>>>> feature >> >>>>> flags. >> >>>>> scan: none requested >> >>>>> config: >> >>>>> >> >>>>> NAME STATE READ WRITE CKSUM >> >>>>> master ONLINE 0 0 0 >> >>>>> raidz3-0 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN01 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN03 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN04 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN05 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN07 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN08 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN09 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN10 ONLINE 0 0 0 >> >>>>> raidz3-1 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN11 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN12 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN13 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN14 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN15 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN16 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN17 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN18 ONLINE 0 0 0 >> >>>>> raidz3-2 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN19 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN20 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN21 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN22 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN23 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN24 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN26 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN27 ONLINE 0 0 0 >> >>>>> raidz3-3 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN29 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN30 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN32 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN33 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN35 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN36 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN37 ONLINE 0 0 0 >> >>>>> multipath/SATA_LUN38 ONLINE 0 0 0 >> >>>>> logs >> >>>>> multipath/SATA_LUN06 ONLINE 0 0 0 >> >>>>> cache >> >>>>> multipath/SATA_LUN02 ONLINE 0 0 0 >> >>>>> >> >>>>> errors: No known data errors >> >>>>> pool: tank >> >>>>> state: ONLINE >> >>>>> status: The pool is formatted using a legacy on-disk format. The >> pool >> >>>>> can >> >>>>> still be used, but some features are unavailable. >> >>>>> action: Upgrade the pool using 'zpool upgrade'. Once this is done, >> the >> >>>>> pool will no longer be accessible on software that does not >> >>>>> support >> >>>>> feature >> >>>>> flags. >> >>>>> scan: none requested >> >>>>> config: >> >>>>> >> >>>>> NAME STATE READ WRITE CKSUM >> >>>>> tank ONLINE 0 0 0 >> >>>>> mirror-0 ONLINE 0 0 0 >> >>>>> da34p3 ONLINE 0 0 0 >> >>>>> da35p3 ONLINE 0 0 0 >> >>>>> >> >>>>> errors: No known data errors >> >>>>> >> >>>>> while compiling a kernel, buildworld worked and installed ok >> >>>>> >> >>>>> lex -t >> >>>>> /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/**** >> >>>>> aicasm/aicasm_scan.l >> >>>>> >> >>>>> aicasm_scan.c >> >>>>>> >> >>>>>> lex -t -Pmm >> >>>>> /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/**** >> >>>>> >> >>>>> aicasm/aicasm_macro_scan.l >> >>>>> >> >>>>> aicasm_macro_scan.c >> >>>>>> >> >>>>>> rm -f .depend_aicasm >> >>>>> mkdep -f .depend_aicasm -a -I. >> >>>>> -I/usr/src/sys/modules/****aic7xxx/aicasm/../../../dev/**** >> >>>>> aic7xxx/aicasm >> >>>>> -std=gnu99 >> >>>>> /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/**** >> >>>>> aicasm/aicasm.c >> >>>>> /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/**** >> >>>>> >> >>>>> aicasm/aicasm_symbol.c >> >>>>> aicasm_gram.c aicasm_macro_gram.c aicasm_scan.c aicasm_macro_scan.c >> >>>>> Out of file descriptors >> >>>>> *** [.depend_aicasm] Error code 2 >> >>>>> >> >>>>> Stop in /usr/src/sys/modules/aic7xxx/****aicasm. >> >>>>> >> >>>>> *** [buildkernel] Error code 1 >> >>>>> >> >>>>> Stop in /usr/src. >> >>>>> *** [buildkernel] Error code 1 >> >>>>> >> >>>>> >> >>>> >> >>>> Do you know what a file descriptor is? >> >>>> >> >>>> There is something about it here: http://www.freebsd.org/doc/en/**** >> >> >>>> books/handbook/configtuning-****kernel-limits.html> >>>> >> freebsd.org/doc/en/books/**handbook/configtuning-kernel-**limits.html< >> http://www.freebsd.org/doc/en/books/handbook/configtuning-kernel-limits.html >> >>(search >> >>>> for 'file descriptor') >> >>>> Or more info in here: https://www.google.nl/search?****< >> https://www.google.nl/search?**> >> >>>> q=freebsd+Out+of+file+****descriptors> >>>> google.nl/search?q=freebsd+**Out+of+file+descriptors< >> https://www.google.nl/search?q=freebsd+Out+of+file+descriptors> >> >>>>> >> >>>> >> >>>> >> >>>> It mainly says that you have more files open than your system is >> >>>> configured to allow. This is often produced by a bug in a program >> which >> >>>> does not close some files properly. >> >>>> >> >>>> >> >>>> Yes I do.... but this is a brand new zfsonroot with barely any data >> on >> >>> it >> >>> and >> >>> >> >>> sysctl -a | grep kern.openfiles >> >>> kern.openfiles: 68 >> >>> root@:/master/builder # sysctl -a | grep kern.maxfiles >> >>> kern.maxfiles: 24600 >> >>> kern.maxfilesperproc: 11095 >> >>> >> >>> the openfiles and maxfiles seems to be plenty, but im out of >> descriptors, >> >>> i >> >>> used to see this back in the 4.x days when you could format >> >>> ufs with larger inodes, but zfs ??? really? >> >>> >> >> >> >> >> >> Is this kern.openfiles when idle or during your buildkernel? >> >> Did you check 'ulimit -a'? >> >> You say installworld worked ok while you are just now doing >> buildkernel. >> >> Are your kernel and world out of sync? >> >> >> >> >> > well, i was upgrading FreeBSD 9-RELEASE to CURRENT, did a make >> buildworld, >> > then a make installworld, when i went to do a make buildkernel >> > thats the error i got, so id say, yes it is "out of sync" >> >> >> Wow wow wow, hold on a sec here. >> >> You're installing your new world before the kernel ? >> >> >> I've had no trouble upgrading several boxes from 8-STABLE to >> 10.0-CURRENT, following the regular procedure as described at: >> http://www.freebsd.org/doc/handbook/makeworld.html >> >> >> Note that the correct order is: >> - buildworld (or kernel-toolchain if you only want to test the kernel and >> not do the whole world) >> - buildkernel >> - installkernel >> - reboot >> - mergemaster -p (preferably in single user, I've always run it multiuser >> w/o problems) >> - installworld >> - mergemaster >> - reboot >> >> You don't want to run the new world on the old kernel. >> > > yeah i know better, seems when i went to install the old kernel the build > had failed, i can get through it from here > > it was odd because the CURRENT kernel failed to build with a yyparse > error, until after i had completed a buildworld > > the other node i did is fine.......... > > > > Perhaps their sources weren't at the same revision level ? > > Anyway, you wanna reinstall a 9.x world from the ISO now, then update > sources and redo things in order ;) > > As a sidenote, if your root is on UFS, you can use nextboot to > failsafe-try your new kernel. > actually there is no need to revert to 9x but thanks, ill just remount the pool read write and do a make installkernel now and reboot