From owner-freebsd-fs@FreeBSD.ORG Wed May 15 12:46:33 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 2E3A6443 for ; Wed, 15 May 2013 12:46:33 +0000 (UTC) (envelope-from outbackdingo@gmail.com) Received: from mail-ob0-x22e.google.com (mail-ob0-x22e.google.com [IPv6:2607:f8b0:4003:c01::22e]) by mx1.freebsd.org (Postfix) with ESMTP id E6BD2A1B for ; Wed, 15 May 2013 12:46:32 +0000 (UTC) Received: by mail-ob0-f174.google.com with SMTP id un3so1912521obb.33 for ; Wed, 15 May 2013 05:46:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=dAr8NSyBwa4ewhD2X9ARR+HAP9erBQNU0fRpGXS2S7Y=; b=JDi9s9qgMXNGraQRWIwrODp2BW8v7IMAWSuX9szuApF+akUmh5E6WZwrcyJA36n5FI N73VRZqjNR3/i6pKqUqhkLXzlisH3erG4oMvFZkfHBUx7dWBTBERINrLf4FFRY9Det6A db2Xpzxz5ugkd0FGE7wAhdDBWa6lHzHPWsMQaZoBhBE7LOG0bcSyRpBWuPsL9471Q8+V WYIdD3L4LHGUTvXTToHjkxvFdfc54aNxuPk5XGiIcLuCkcclf1kUAldnjcN4wrZlkuOa Zzh9pG3+TejyzeYu4bm7vq5Fd4txmmAZz2JhrjqbCiOkEl2As5PFv73UEG117ChGTSUI rVTQ== MIME-Version: 1.0 X-Received: by 10.182.46.228 with SMTP id y4mr17273243obm.28.1368621992438; Wed, 15 May 2013 05:46:32 -0700 (PDT) Received: by 10.76.96.49 with HTTP; Wed, 15 May 2013 05:46:32 -0700 (PDT) In-Reply-To: References: Date: Wed, 15 May 2013 08:46:32 -0400 Message-ID: Subject: Re: FreeBSD 9-RELEASE zpool Out of File Descriptors ?? From: Outback Dingo To: Ronald Klop Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 May 2013 12:46:33 -0000 On Wed, May 15, 2013 at 8:34 AM, Ronald Klop wrote: > On Wed, 15 May 2013 14:27:10 +0200, Outback Dingo > wrote: > > On Wed, May 15, 2013 at 8:22 AM, Ronald Klop > >**wrote: >> >> On Wed, 15 May 2013 14:13:32 +0200, Outback Dingo < >>> outbackdingo@gmail.com> >>> wrote: >>> >>> So it seems a new deployment we just built with zfsonroot mirror and a >>> >>>> 48TB >>>> master pool is already out of File Descriptors??? >>>> >>>> pool: master >>>> state: ONLINE >>>> status: The pool is formatted using a legacy on-disk format. The pool >>>> can >>>> still be used, but some features are unavailable. >>>> action: Upgrade the pool using 'zpool upgrade'. Once this is done, the >>>> pool will no longer be accessible on software that does not >>>> support >>>> feature >>>> flags. >>>> scan: none requested >>>> config: >>>> >>>> NAME STATE READ WRITE CKSUM >>>> master ONLINE 0 0 0 >>>> raidz3-0 ONLINE 0 0 0 >>>> multipath/SATA_LUN01 ONLINE 0 0 0 >>>> multipath/SATA_LUN03 ONLINE 0 0 0 >>>> multipath/SATA_LUN04 ONLINE 0 0 0 >>>> multipath/SATA_LUN05 ONLINE 0 0 0 >>>> multipath/SATA_LUN07 ONLINE 0 0 0 >>>> multipath/SATA_LUN08 ONLINE 0 0 0 >>>> multipath/SATA_LUN09 ONLINE 0 0 0 >>>> multipath/SATA_LUN10 ONLINE 0 0 0 >>>> raidz3-1 ONLINE 0 0 0 >>>> multipath/SATA_LUN11 ONLINE 0 0 0 >>>> multipath/SATA_LUN12 ONLINE 0 0 0 >>>> multipath/SATA_LUN13 ONLINE 0 0 0 >>>> multipath/SATA_LUN14 ONLINE 0 0 0 >>>> multipath/SATA_LUN15 ONLINE 0 0 0 >>>> multipath/SATA_LUN16 ONLINE 0 0 0 >>>> multipath/SATA_LUN17 ONLINE 0 0 0 >>>> multipath/SATA_LUN18 ONLINE 0 0 0 >>>> raidz3-2 ONLINE 0 0 0 >>>> multipath/SATA_LUN19 ONLINE 0 0 0 >>>> multipath/SATA_LUN20 ONLINE 0 0 0 >>>> multipath/SATA_LUN21 ONLINE 0 0 0 >>>> multipath/SATA_LUN22 ONLINE 0 0 0 >>>> multipath/SATA_LUN23 ONLINE 0 0 0 >>>> multipath/SATA_LUN24 ONLINE 0 0 0 >>>> multipath/SATA_LUN26 ONLINE 0 0 0 >>>> multipath/SATA_LUN27 ONLINE 0 0 0 >>>> raidz3-3 ONLINE 0 0 0 >>>> multipath/SATA_LUN29 ONLINE 0 0 0 >>>> multipath/SATA_LUN30 ONLINE 0 0 0 >>>> multipath/SATA_LUN32 ONLINE 0 0 0 >>>> multipath/SATA_LUN33 ONLINE 0 0 0 >>>> multipath/SATA_LUN35 ONLINE 0 0 0 >>>> multipath/SATA_LUN36 ONLINE 0 0 0 >>>> multipath/SATA_LUN37 ONLINE 0 0 0 >>>> multipath/SATA_LUN38 ONLINE 0 0 0 >>>> logs >>>> multipath/SATA_LUN06 ONLINE 0 0 0 >>>> cache >>>> multipath/SATA_LUN02 ONLINE 0 0 0 >>>> >>>> errors: No known data errors >>>> pool: tank >>>> state: ONLINE >>>> status: The pool is formatted using a legacy on-disk format. The pool >>>> can >>>> still be used, but some features are unavailable. >>>> action: Upgrade the pool using 'zpool upgrade'. Once this is done, the >>>> pool will no longer be accessible on software that does not >>>> support >>>> feature >>>> flags. >>>> scan: none requested >>>> config: >>>> >>>> NAME STATE READ WRITE CKSUM >>>> tank ONLINE 0 0 0 >>>> mirror-0 ONLINE 0 0 0 >>>> da34p3 ONLINE 0 0 0 >>>> da35p3 ONLINE 0 0 0 >>>> >>>> errors: No known data errors >>>> >>>> while compiling a kernel, buildworld worked and installed ok >>>> >>>> lex -t >>>> /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/**** >>>> aicasm/aicasm_scan.l >>>> >>>> aicasm_scan.c >>>>> >>>>> lex -t -Pmm >>>> /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/**** >>>> >>>> aicasm/aicasm_macro_scan.l >>>> >>>> aicasm_macro_scan.c >>>>> >>>>> rm -f .depend_aicasm >>>> mkdep -f .depend_aicasm -a -I. >>>> -I/usr/src/sys/modules/****aic7xxx/aicasm/../../../dev/**** >>>> aic7xxx/aicasm >>>> -std=gnu99 >>>> /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/**** >>>> aicasm/aicasm.c >>>> /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/**** >>>> >>>> aicasm/aicasm_symbol.c >>>> aicasm_gram.c aicasm_macro_gram.c aicasm_scan.c aicasm_macro_scan.c >>>> Out of file descriptors >>>> *** [.depend_aicasm] Error code 2 >>>> >>>> Stop in /usr/src/sys/modules/aic7xxx/****aicasm. >>>> >>>> *** [buildkernel] Error code 1 >>>> >>>> Stop in /usr/src. >>>> *** [buildkernel] Error code 1 >>>> >>>> >>> >>> Do you know what a file descriptor is? >>> >>> There is something about it here: http://www.freebsd.org/doc/en/**** >>> books/handbook/configtuning-****kernel-limits.html>> freebsd.org/doc/en/books/**handbook/configtuning-kernel-**limits.html>(search >>> for 'file descriptor') >>> Or more info in here: https://www.google.nl/search?**** >>> q=freebsd+Out+of+file+****descriptors>> google.nl/search?q=freebsd+**Out+of+file+descriptors >>> > >>> >>> >>> It mainly says that you have more files open than your system is >>> configured to allow. This is often produced by a bug in a program which >>> does not close some files properly. >>> >>> >>> Yes I do.... but this is a brand new zfsonroot with barely any data on >> it >> and >> >> sysctl -a | grep kern.openfiles >> kern.openfiles: 68 >> root@:/master/builder # sysctl -a | grep kern.maxfiles >> kern.maxfiles: 24600 >> kern.maxfilesperproc: 11095 >> >> the openfiles and maxfiles seems to be plenty, but im out of descriptors, >> i >> used to see this back in the 4.x days when you could format >> ufs with larger inodes, but zfs ??? really? >> > > > Is this kern.openfiles when idle or during your buildkernel? > Did you check 'ulimit -a'? > You say installworld worked ok while you are just now doing buildkernel. > Are your kernel and world out of sync? > > well, i was upgrading FreeBSD 9-RELEASE to CURRENT, did a make buildworld, then a make installworld, when i went to do a make buildkernel thats the error i got, so id say, yes it is "out of sync" my ulimits are ulimit -a cpu time (seconds, -t) unlimited file size (512-blocks, -f) unlimited data seg size (kbytes, -d) 33554432 stack size (kbytes, -s) 524288 core file size (512-blocks, -c) unlimited max memory size (kbytes, -m) unlimited locked memory (kbytes, -l) unlimited max user processes (-u) 5547 open files (-n) 11095 virtual mem size (kbytes, -v) unlimited swap limit (kbytes, -w) unlimited sbsize (bytes, -b) unlimited pseudo-terminals (-p) unlimited > Ronald. >