Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 30 May 2013 12:56:54 +0200
From:      Palle Girgensohn <girgen@FreeBSD.org>
To:        Kirk McKusick <mckusick@mckusick.com>
Cc:        freebsd-fs@FreeBSD.org, Dan Thomas <godders@gmail.com>, Jeff Roberson <jroberson@jroberson.net>, Julian Akehurst <julian@pingpong.se>
Subject:   Re: leaking lots of unreferenced inodes (pg_xlog files?)
Message-ID:  <51A73076.8020609@FreeBSD.org>
In-Reply-To: <alpine.BSF.2.00.1305061329480.2005@desktop>
References:  <201305062250.r46MoWkx091499@chez.mckusick.com> <alpine.BSF.2.00.1305061329480.2005@desktop>

next in thread | previous in thread | raw e-mail | index | archive | help

[-- Attachment #1 --]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello again!

I have now remounted the postgresql filesystem on a test server that
experiences the same problem. The production server is not remounted
yet, we're planning that in a weeks time approximately, but I though I
could gain som time by running the suggested procedure on the test box.

The base problem was this:

# df -h /pgsql ; du -hxs /pgsql
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/da2s1d    128G    101G     17G    86%    /pgsql
 82G	/pgsql

df says 101 GB used, but du only finds 82 GB, and fstat cannot find any
open files that are unreferenced in the file system. Stopping postgresql
does not help. It seems the OS is leaking inode references.

FreeBSD 9.1, postgresql 9.2.3 from port.

I ran the suggested commans (in attached diskspacecheck) before stopping
postgresql (before.log), after stopping postgresql but before unmount
/pgsql (before2.log), and then i unmounted /pgsql (had to run umount -f
/pgsql, and it took about 20 seconds). I did not enter single-user mode,
since I really did not have to this time (On the production server, the
disk is /usr, so that will require more shutting down...)

I've attach the logs here. Hope it helps!

The commands run in diskspaccheck are
#! /bin/sh
df -ih /pgsql
vmstat -z
vmstat -m
sysctl debug
fstat -f /pgsql

as suggested by Kirk.








Jeff Roberson skrev:
> On Mon, 6 May 2013, Kirk McKusick wrote:
> 
>>> Date: Sat, 04 May 2013 15:34:33 +0200 From: Palle Girgensohn
>>> <girgen@freebsd.org> To: Kirk McKusick <mckusick@mckusick.com> 
>>> Subject: Re: leaking lots of unreferenced inodes (pg_xlog
>>> files?), maybe after moving tables and indexes to tablespace on
>>> different volume Cc: freebsd-fs@freebsd.org, Jeff Roberson
>>> <jroberson@jroberson.net>
>>> 
>>> Hi,
>>> 
>>> Just a quick ping on this issue, it is still happening and we are
>>> slowly filling up the disk again. Is seems like we will have to
>>> plan a remount within a month given the current graphs. I will be
>>> back with info once we have remounted it, running you suggested
>>> scripts before and after.
>>> 
>>> Is there anything else I can do to get more debug information?
>>> 
>>> Regards, Palle
>> 
>> In addition to the things that I mentioned previously please also
>> run `vmstat -z' before and after the unmount/mount.
>> 
>> Jeff, do you have any other things that you think he should run?
>> 
>> Kirk
>> 
> 
> Remounting fixes it?  Does it require a fsck?  I don't have enough 
> context to comment here.  The vmstat will tell us if it's a leak in
> a memory reference but that should blow up memory before is saturates
> a volume unless the volume is very small.  How big is the volume?
> How much memory do you have?  The inodes show up as used in df -i?
> How many more than find sees?  What is kern.maxvnodes and
> vfs.numvnodes?
> 
> Jeff
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJRpzB2AAoJEIhV+7FrxBJDVIYIAK8a2zAx6pUlDn1CDj1AlSb3
jvNRh/YgTUJBj0JSkh93uRqi8A6sy1gGz6PHqOhUE4uN+nNXCIqkYvPkLJ7C8zRW
O6LCJ9nSMud6woGpeBRz8V3bu2RbHZMOATZ2f1dyzFPOyB8nU02zsoK3yOojUsWk
PpCGIpSZafKfP1A98wKNxFBoo59+HxePqbAEQeP4Cl5+/Hudnj6hwBdB0DsZkhkU
4vSVEzQ360iiQDFTurxIv+CARvtrnhYC+iFxPsQftevEbhlRH5z9YsUZkR5YVms/
h+Y551yqB4NrsRqcw0qQOf9lNb10PK74z7sCrizsIKFonYkXGQSv16AXWXnG95I=
=LvOT
-----END PGP SIGNATURE-----

[-- Attachment #2 --]
Script started on Wed May 29 18:54:13 2013
./diskspacecheck.sh
Filesystem     Size    Used   Avail Capacity iused ifree %iused  Mounted on
/dev/da2s1d    128G    101G     17G    86%     27k   17M    0%   /pgsql
ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP

UMA Kegs:               208,      0,      86,      16,      86,   0,   0
UMA Zones:             1408,      0,      86,       0,      86,   0,   0
UMA Slabs:              568,      0,    7023,     320,  609264,   0,   0
UMA RCntSlabs:          568,      0,    4335,     852,  387257,   0,   0
UMA Hash:               256,      0,       0,       0,       3,   0,   0
16 Bucket:              152,      0,      25,      75,     209,   0,   0
32 Bucket:              280,      0,      47,     149,     339,   6,   0
64 Bucket:              536,      0,      78,      76,     752,  90,   0
128 Bucket:            1048,      0,     773,     244, 1511895,14953566,   0
VM OBJECT:              232,      0,   80289,  105743,137619934,   0,   0
MAP:                    232,      0,       7,      25,       7,   0,   0
KMAP ENTRY:             120, 1128710,      79,   51133,1725903322,   0,   0
MAP ENTRY:              120,      0,    2067,    5900,293018731,   0,   0
fakepg:                 120,      0,       0,       0,       0,   0,   0
mt_zone:               4112,      0,     326,       1,     326,   0,   0
16:                      16,      0,    2817,    1551,14747445,   0,   0
32:                      32,      0,    2956,    1387,48861077,   0,   0
64:                      64,      0,    5398,    3282,80610637,   0,   0
128:                    128,      0,    9899,    4862,667748476,   0,   0
256:                    256,      0,    1000,    5225,2814861913,   0,   0
512:                    512,      0,    2788,    2000,17565459,   0,   0
1024:                  1024,      0,      71,    1209, 2134386,   0,   0
2048:                  2048,      0,    5657,    1207, 4009107,   0,   0
4096:                  4096,      0,     451,    1021, 5175688,   0,   0
Files:                   80,      0,     927,   10278,193059740,   0,   0
TURNSTILE:              136,      0,    1939,     121,    1948,   0,   0
umtx pi:                 96,      0,       0,       0,       0,   0,   0
MAC labels:              40,      0,       0,       0,       0,   0,   0
PROC:                  1184,      0,      68,    1417, 4406945,   0,   0
THREAD:                1128,      0,    1549,     389,    1951,   0,   0
SLEEPQUEUE:              80,      0,    1939,     265,    1948,   0,   0
VMSPACE:                392,      0,      49,    1541, 4406928,   0,   0
cpuset:                  72,      0,      85,      65,      85,   0,   0
audit_record:           960,      0,       0,       0,       0,   0,   0
mbuf_packet:            256,      0,    4080,    1296,688503852,   0,   0
mbuf:                   256,      0,    1021,    1973,1942026536,   0,   0
mbuf_cluster:          2048,  25600,    5376,    1894, 1693184,   0,   0
mbuf_jumbo_page:       4096,  12800,       0,     700,83316546,   0,   0
mbuf_jumbo_9k:         9216,   6400,       0,       0,       0,   0,   0
mbuf_jumbo_16k:       16384,   3200,       0,       0,       0,   0,   0
mbuf_ext_refcnt:          4,      0,       0,       0,       0,   0,   0
g_bio:                  232,      0,       0,    6064,2877087500,   0,   0
ttyinq:                 160,      0,     150,     378,    2490,   0,   0
ttyoutq:                256,      0,      80,     325,    1328,   0,   0
ata_request:            328,      0,       0,      84,      32,   0,   0
ata_composite:          336,      0,       0,       0,       0,   0,   0
VNODE:                  480,      0,  224754,   34526,153776522,   0,   0
VNODEPOLL:              112,      0,       0,     132,       6,   0,   0
NAMEI:                 1024,      0,       0,    1212,418886439,   0,   0
S VFS Cache:            108,      0,  232208,   43870,158049447,   0,   0
STS VFS Cache:          148,      0,       0,       0,       0,   0,   0
L VFS Cache:            328,      0,     681,    4959, 7376729,   0,   0
LTS VFS Cache:          368,      0,       0,       0,       0,   0,   0
NCLNODE:                568,      0,       0,       0,       0,   0,   0
DIRHASH:               1024,      0,    1971,     613, 2981873,   0,   0
Mountpoints:            792,      0,       8,      22,       8,   0,   0
pipe:                   728,      0,      23,    1207, 3848927,   0,   0
ksiginfo:               112,      0,    1474,    1430, 9770473,   0,   0
itimer:                 344,      0,       1,      21,       1,   0,   0
KNOTE:                  128,      0,       0,    1711, 1098021,   0,   0
socket:                 680, 100002,      52,    1406, 9367833,   0,   0
ipq:                     56,    819,       0,       0,       0,   0,   0
udp_inpcb:              392, 100000,      13,    1417, 7328189,   0,   0
udpcb:                   16, 100128,      13,    1835, 7328189,   0,   0
tcp_inpcb:              392, 100000,      23,    1737,  652060,   0,   0
tcpcb:                  976, 100000,      23,    1749,  652060,   0,   0
tcptw:                   72,  20000,       0,     550,    3251,   0,   0
syncache:               152,  15375,       0,     625,  505439,   0,   0
hostcache:              136,  15372,       3,     445,    1819,   0,   0
tcpreass:                40,   1680,       0,    1596,   21690,   0,   0
sackhole:                32,      0,       0,     909,     875,   0,   0
sctp_ep:               1376, 100000,       0,       0,       0,   0,   0
sctp_asoc:             2288,  40000,       0,       0,       0,   0,   0
sctp_laddr:              48,  80064,       0,     288,       6,   0,   0
sctp_raddr:             704,  80000,       0,       0,       0,   0,   0
sctp_chunk:             136, 400008,       0,       0,       0,   0,   0
sctp_readq:             104, 400032,       0,       0,       0,   0,   0
sctp_stream_msg_out:    112, 400026,       0,       0,       0,   0,   0
sctp_asconf:             40, 400008,       0,       0,       0,   0,   0
sctp_asconf_ack:         48, 400032,       0,       0,       0,   0,   0
ripcb:                  392, 100000,       0,     100,     122,   0,   0
unpcb:                  240, 100000,      15,    1313, 1387456,   0,   0
rtentry:                200,      0,      23,      91,      23,   0,   0
selfd:                   56,      0,    2035,    1304,300180470,   0,   0
SWAPMETA:               288, 116519,      47,     551, 1053562,   0,   0
FFS inode:              168,      0,  224716,   41286,153774289,   0,   0
FFS1 dinode:            128,      0,       0,       0,       0,   0,   0
FFS2 dinode:            256,      0,  224716,   39989,153774283,   0,   0

         Type InUse MemUse HighUse Requests  Size(s)
CAM dev queue     4     1K       -        4  128
  md_sii_data     0     0K       -       38  512
      CAM XPT   571  1045K       -      943  16,32,64,128,256,1024,2048
       isadev     8     1K       -        8  128
         UART     6     4K       -        6  16,512,1024
     acpiintr     1     1K       -        1  64
         cdev     7     2K       -        7  256
       acpica  1451   139K       -  6308502  16,32,64,128,256,512,1024
        sigio     1     1K       -        1  64
     filedesc   120   142K       -  4903990  16,32,64,128,256,512,1024,2048,4096
         kenv    78    11K       -       89  16,32,64,128
       kqueue     0     0K       -  1056987  256,512,2048
    proc-args    49     3K       -  6967355  16,32,64,128,256
        hhook     2     1K       -        2  128
     acpitask     1     2K       -        1  2048
      ithread    99    16K       -       99  32,128,256
    CAM queue    19     7K       -      407  16,32,64,128,256,512,2048
       KTRACE   100    13K       -      100  128
       linker   158    11K       -      160  16,32,64,512
        lockf    20     3K       -    23976  64,128
   loginclass     2     1K       -    59061  64
       ip6ndp    12     1K       -       14  64,128
         temp    51     2K       -  8217650  16,32,64,128,256,512,1024,2048,4096
       devbuf  8648 41889K       -    10143  16,32,64,128,256,512,1024,2048,4096
       module   477    60K       -      477  128
    ciss_data    15    14K       -     5921  16,128,256,512,1024,2048,4096
     mtx_pool     2    16K       -        2  
       USBdev    34     8K       -       44  64,128,512,4096
          USB    52    31K       -       66  16,32,64,128,256,2048,4096
     pmchooks     1     1K       -        1  128
      subproc  1556  1020K       -  4408434  512,4096
         proc     2    16K       -        2  
      session    43     6K       -  3550674  128
         pgrp    47     6K       -  3555087  128
         cred    60    10K       - 32876160  64,256
      uidinfo     6     3K       -   186859  128,2048
       plimit    17     5K       -   770696  256
      ata_pci     1     1K       -        1  64
      scsi_cd     0     0K       -       10  16
    sysctltmp     0     0K       -    14220  16,32,64,128,4096
    sysctloid  4978   251K       -     5360  16,32,64,128
       sysctl     0     0K       -  7955789  16,32,64
      tidhash     1    16K       -        1  
      callout     7 14336K       -        7  
         umtx  3876   485K       -     3894  128
     p1003.1b     1     1K       -        1  16
         SWAP     2   549K       -        2  64
       bus-sc   113   781K       -     5234  16,32,64,128,256,512,1024,2048,4096
          bus  1282   109K       -     8626  16,32,64,128,256,1024
      devstat    12    25K       -       12  32,4096
 eventhandler    77     6K       -       77  64,128
      acpisem    19     3K       -       19  128
         kobj   330  1320K       -      626  4096
      CAM SIM     5     2K       -        5  256
      Per-cpu     1     1K       -        1  32
   CAM periph    10     3K       -       54  16,32,64,128,256
         rman   248    26K       -      613  16,32,128
         sbuf     1     1K       -     2060  16,32,64,128,256,512,1024,2048,4096
      entropy  1024    64K       -     1024  64
       ctlmem  5062 10113K       -     5062  128,2048
        stack     0     0K       -        2  256
    taskqueue    15     2K       -       15  16,32,128
       Unitno    14     1K       -   596716  32,64
          iov     0     0K       -  1560732  16,32,64,128,256,512
       select  1455   182K       -     1455  128
     ioctlops     0     0K       -  8497058  16,32,64,128,256,512,1024,2048,4096
          msg     4    30K       -        4  2048,4096
          sem     4   196K       -        4  
          shm    21    60K       -  3349278  2048
          tty    24    24K       -      148  1024,2048
          pts     3     1K       -      125  256
     mbuf_tag     0     0K       - 43344652  32,128
        shmfd     1     8K       -        1  
          pcb    23   157K       -   629088  16,32,128,1024,2048,4096
       soname     5     1K       - 52229622  16,32,128
          acl     0     0K       -   144101  4096
       biobuf     1     2K       -    14683  2048
     vfscache     1  8192K       -        1  
   cl_savebuf     0     0K       - 101708382  64,128
     vfs_hash     1  4096K       -        1  
       DEVFS1   116    58K       -      261  512
       vnodes     2     1K       -        4  64,256
       DEVFS3   132    33K       -      281  256
        mount   106     5K       -      176  16,32,64,128,256
  vnodemarker     0     0K       -  5062311  512
          BPF     9     2K       -        9  128
  ether_multi    56     3K       -       66  16,32,64
       ifaddr    87    22K       -       87  32,64,128,256,512,4096
        ifnet    10    19K       -       10  128,2048
        clone     6    24K       -        6  4096
       arpcom     2     1K       -        2  16
      lltable    31    13K       -    58482  256,512
        DEVFS    17     1K       -       20  16,128
       DEVFSP     1     1K       -        2  64
     routetbl    44     6K       -  1464202  32,64,128,256,512
         igmp     9     3K       -        9  256
     in_multi     3     1K       -        3  256
    sctp_iter     0     0K       -        4  256
     sctp_ifn     3     1K       -        3  128
     sctp_ifa     7     1K       -        7  128
     sctp_vrf     1     1K       -        1  64
    sctp_a_it     0     0K       -        4  16
    hostcache     1    28K       -        1  
     syncache     1    96K       -        1  
    in6_multi    28     4K       -       28  32,256
          mld     9     2K       -        9  128
          rpc     2     1K       -        2  256
audit_evclass   179     6K       -      218  32
     savedino     0     0K       -  3083223  256
     freework     1     1K       - 80548685  16,128
    newdirblk     0     0K       -    18337  64
       dirrem     1     1K       - 13082996  128
        mkdir     0     0K       -    29436  128
       diradd     1     1K       - 13091229  128
     freefile     1     1K       -  6713927  64
     freeblks     0     0K       -  6611294  128
     freefrag     0     0K       - 427582244  128
     indirdep     1     1K       -  6251906  128
       newblk   407   614K       - 2781862141  256
    bmsafemap    10    11K       -  8938352  256
     inodedep     5  4098K       -  7832059  512
      pagedep     2   513K       -   139118  256
  ufs_dirhash  1462   372K       -   401644  16,32,64,128,256,512,1024
    ufs_mount    18    69K       -       18  512,2048,4096
    vm_pgdata     2   129K       -        2  128
      UMAHash     3    22K       -       13  512,1024,2048,4096
      memdesc     1     4K       -        1  4096
     atkbddev     2     1K       -        2  64
    pfs_nodes    21     6K       -       21  256
  pfs_vncache     1     1K       -      260  64
       ctlblk   200  1600K       -      200  
         GEOM   127    40K       -     1312  16,32,64,128,256,512,1024,2048
      ramdisk     1  4096K       -        1  
      acpidev    30     2K       -       30  64
      ctlpool   532   142K       -      532  32,512
       kbdmux     7    18K       -        8  16,512,1024,2048
       apmdev     1     1K       -        1  128
   madt_table     0     0K       -        1  4096
       feeder     7     1K       -        7  32
      scsi_da     0     0K       -      266  16
     pci_link    16     2K       -       16  16,128
    raid_data     0     0K       -      234  32,128,256
      io_apic     1     2K       -        1  2048
          MCA     8     1K       -        8  128
          msi     3     1K       -        3  128
     nexusdev     3     1K       -        3  16
md_nvidia_data     0     0K       -       38  512
debug.acpi.suspend_bounce: 0
debug.acpi.reset_clock: 1
debug.acpi.interpreter_slack: 1
debug.acpi.enable_debug_objects: 0
debug.acpi.acpi_ca_version: 20110527
debug.acpi.cpu_unordered: 0
debug.acpi.ec.timeout: 750
debug.acpi.ec.polled: 0
debug.acpi.ec.burst: 0
debug.acpi.batt.batt_sleep_ms: 0
debug.acpi.resume_beep: 0
debug.firewire_debug: 0
debug.fwmem_debug: 0
debug.if_fwe_debug: 0
debug.if_fwip_debug: 0
debug.ipw: 0
debug.iwi: 0
debug.mddebug: 0
debug.wpi: 0
debug.elf64_legacy_coredump: 0
debug.bootverbose: 0
debug.boothowto: 0
debug.cpufreq.verbose: 0
debug.cpufreq.lowest: 0
debug.fail_point.sysctl_running: off
debug.fail_point.buf_pressure: off
debug.fail_point.nlm_deny_grant: off
debug.adaptive_machine_arch: 1
debug.sizeof.cdev_priv: 376
debug.sizeof.cdev: 288
debug.sizeof.g_bioq: 56
debug.sizeof.g_consumer: 96
debug.sizeof.g_provider: 136
debug.sizeof.g_geom: 160
debug.sizeof.g_class: 160
debug.sizeof.kinfo_proc: 1088
debug.sizeof.buf: 608
debug.sizeof.bio: 232
debug.sizeof.proc: 1184
debug.sizeof.vnode: 480
debug.sizeof.devstat: 288
debug.sizeof.namecache: 72
debug.osd: 0
debug.trace_on_panic: 1
debug.debugger_on_panic: 1
debug.ncores: 5
debug.to_avg_mpcalls: 745
debug.to_avg_lockcalls: 245
debug.to_avg_gcalls: 496
debug.to_avg_depth: 1928
debug.umtx.umtx_pi_allocated: 0
debug.clocktime: 0
debug.kdb.alt_break_to_debugger: 0
debug.kdb.break_to_debugger: 0
debug.kdb.trap_code: 0
debug.kdb.trap: 0
debug.kdb.panic: 0
debug.kdb.enter: 0
debug.kdb.current: 
debug.rman_debug: 0
debug.iosize_max_clamp: 1
debug.disablefullpath: 0
debug.disablecwd: 0
debug.vfscache: 1
debug.numcachehv: 25234
debug.numcache: 232889
debug.numneg: 12146
debug.nchash: 1048575
debug.vnlru_nowhere: 0
debug.rush_requests: 370281
debug.if_tun_debug: 0
debug.nlm_debug: 0
debug.fsckcmds: 0
debug.collectsnapstats: 0
debug.snapdebug: 0
debug.dopersistence: 0
debug.softdep.cleanup_failures: 369634
debug.softdep.cleanup_retries: 826
debug.softdep.cleanup_high_delay: 14
debug.softdep.cleanup_inorequests: 0
debug.softdep.cleanup_blkrequests: 369690
debug.softdep.jwait_newblk: 0
debug.softdep.jwait_inode: 0
debug.softdep.jwait_freeblks: 0
debug.softdep.jwait_filepage: 0
debug.softdep.journal_wait: 0
debug.softdep.journal_min: 0
debug.softdep.journal_low: 0
debug.softdep.jnewblk_rollback: 0
debug.softdep.jaddref_rollback: 0
debug.softdep.dir_entry: 195659
debug.softdep.direct_blk_ptrs: 339021
debug.softdep.inode_bitmap: 2503730
debug.softdep.indir_blk_ptrs: 26881
debug.softdep.sync_limit_hit: 591
debug.softdep.ino_limit_hit: 0
debug.softdep.blk_limit_hit: 369220
debug.softdep.ino_limit_push: 0
debug.softdep.blk_limit_push: 369690
debug.softdep.worklist_push: 2156
debug.softdep.maxindirdeps: 50
debug.softdep.tickdelay: 2
debug.softdep.max_softdeps: 2352308
debug.softdep.write.jfsync: 0
debug.softdep.write.jtrunc: 0
debug.softdep.write.sbdep: 0
debug.softdep.write.jsegdep: 0
debug.softdep.write.jseg: 0
debug.softdep.write.jfreefrag: 0
debug.softdep.write.jfreeblk: 0
debug.softdep.write.jnewblk: 0
debug.softdep.write.jmvref: 0
debug.softdep.write.jremref: 0
debug.softdep.write.jaddref: 0
debug.softdep.write.freedep: 0
debug.softdep.write.freework: 0
debug.softdep.write.newdirblk: 0
debug.softdep.write.dirrem: 0
debug.softdep.write.mkdir: 0
debug.softdep.write.diradd: 0
debug.softdep.write.freefile: 0
debug.softdep.write.freeblks: 668958
debug.softdep.write.freefrag: 0
debug.softdep.write.allocindir: 2248011316
debug.softdep.write.indirdep: 115439
debug.softdep.write.allocdirect: 74254013
debug.softdep.write.newblk: 0
debug.softdep.write.bmsafemap: 2393906
debug.softdep.write.inodedep: 6801109
debug.softdep.write.pagedep: 320313
debug.softdep.current.jfsync: 0
debug.softdep.current.jtrunc: 0
debug.softdep.current.sbdep: 0
debug.softdep.current.jsegdep: 0
debug.softdep.current.jseg: 0
debug.softdep.current.jfreefrag: 0
debug.softdep.current.jfreeblk: 0
debug.softdep.current.jnewblk: 0
debug.softdep.current.jmvref: 0
debug.softdep.current.jremref: 0
debug.softdep.current.jaddref: 0
debug.softdep.current.freedep: 0
debug.softdep.current.freework: 0
debug.softdep.current.newdirblk: 0
debug.softdep.current.dirrem: 1
debug.softdep.current.mkdir: 0
debug.softdep.current.diradd: 1
debug.softdep.current.freefile: 1
debug.softdep.current.freeblks: 0
debug.softdep.current.freefrag: 0
debug.softdep.current.allocindir: 0
debug.softdep.current.indirdep: 1
debug.softdep.current.allocdirect: 0
debug.softdep.current.newblk: 406
debug.softdep.current.bmsafemap: 9
debug.softdep.current.inodedep: 4
debug.softdep.current.pagedep: 1
debug.softdep.total.jfsync: 0
debug.softdep.total.jtrunc: 0
debug.softdep.total.sbdep: 0
debug.softdep.total.jsegdep: 0
debug.softdep.total.jseg: 0
debug.softdep.total.jfreefrag: 0
debug.softdep.total.jfreeblk: 0
debug.softdep.total.jnewblk: 0
debug.softdep.total.jmvref: 0
debug.softdep.total.jremref: 0
debug.softdep.total.jaddref: 0
debug.softdep.total.freedep: 0
debug.softdep.total.freework: 80548684
debug.softdep.total.newdirblk: 18337
debug.softdep.total.dirrem: 13082996
debug.softdep.total.mkdir: 29436
debug.softdep.total.diradd: 13091229
debug.softdep.total.freefile: 6713927
debug.softdep.total.freeblks: 6611294
debug.softdep.total.freefrag: 427582244
debug.softdep.total.allocindir: 0
debug.softdep.total.indirdep: 6225025
debug.softdep.total.allocdirect: 0
debug.softdep.total.newblk: 2781862140
debug.softdep.total.bmsafemap: 8938351
debug.softdep.total.inodedep: 7832058
debug.softdep.total.pagedep: 139117
debug.dobkgrdwrite: 1
debug.bigcgs: 0
debug.dircheck: 0
debug.psm.pkterrthresh: 2
debug.psm.usecs: 500000
debug.psm.secs: 0
debug.psm.errusecs: 0
debug.psm.errsecs: 2
debug.psm.hz: 20
debug.psm.loglevel: 0
debug.vesa.shadow_rom: 0
debug.fdc.settle: 0
debug.fdc.spec2: 16
debug.fdc.spec1: 175
debug.fdc.retries: 10
debug.fdc.debugflags: 0
debug.fdc.fifo: 8
debug.elf32_legacy_coredump: 0
debug.x86bios.int: 0
debug.x86bios.call: 0
debug.hwpstate_verbose: 0
debug.minidump: 1
USER     CMD          PID   FD MOUNT      INUM MODE         SZ|DV R/W
pgsql    postgres   12885   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres   12885    6 /pgsql   15780331 -rw-------    8192 rw
pgsql    postgres   12885   12 /pgsql   15779866 -rw-------   16384 rw
pgsql    postgres   12869   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres   12869    6 /pgsql   12671024 -rw-------    8192 rw
pgsql    postgres   12788   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres   12788    6 /pgsql   11046014 -rw-------    8192 rw
pgsql    postgres   12788   40 /pgsql   11053157 -rw-------   16384 rw
pgsql    postgres   12747   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres   12747    6 /pgsql   12671024 -rw-------    8192 rw
pgsql    postgres   12730   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres   12730    6 /pgsql   11046014 -rw-------    8192 rw
pgsql    postgres   12730   40 /pgsql   11053157 -rw-------   16384 rw
pgsql    postgres   12730   54 /pgsql   11053304 -rw-------  16777216 rw
pgsql    postgres   12729   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres   12729    6 /pgsql   11046014 -rw-------    8192 rw
pgsql    postgres   12631   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres   12631    6 /pgsql   15780331 -rw-------    8192 rw
pgsql    postgres   12631   12 /pgsql   15779866 -rw-------   16384 rw
pgsql    postgres   12614   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres   12614    6 /pgsql   12671024 -rw-------    8192 rw
pgsql    postgres   12528   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres   12528    6 /pgsql   11046014 -rw-------    8192 rw
pgsql    postgres   12528   40 /pgsql   11053157 -rw-------   16384 rw
pgsql    postgres   12528   54 /pgsql   11053304 -rw-------  16777216 rw
pgsql    postgres   12467   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres   12467    6 /pgsql   11046014 -rw-------    8192 rw
pgsql    postgres   12467   40 /pgsql   11053157 -rw-------   16384 rw
pgsql    postgres   12467   57 /pgsql   11053304 -rw-------  16777216 rw
pgsql    postgres   12134   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres   12134    6 /pgsql   15780331 -rw-------    8192 rw
pgsql    postgres   12134   12 /pgsql   15779866 -rw-------   16384 rw
pgsql    postgres   12134  132 /pgsql   15779886 -rw-------   81920 rw
pgsql    postgres   12134  133 /pgsql   15779880 -rw-------  257138688 rw
pgsql    postgres   12134  135 /pgsql   15779889 -rw-------  75677696 rw
pgsql    postgres   12134  136 /pgsql   11053304 -rw-------  16777216 rw
pgsql    postgres   54086   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres   54086    6 /pgsql   11046014 -rw-------    8192 rw
pgsql    postgres   54086   13 /pgsql   11046096 -rw-------  1179648 rw
pgsql    postgres   54086   14 /pgsql   11053374 -rw-------  16777216 rw
pgsql    postgres   54086   15 /pgsql   11046011 -rw-------  794624 rw
pgsql    postgres   54086   16 /pgsql   11046039 -rw-------  131072 rw
pgsql    postgres   54086   17 /pgsql   11046063 -rw-------  335872 rw
pgsql    postgres   54086   18 /pgsql   11046021 -rw-------   24576 rw
pgsql    postgres   54086   19 /pgsql   11046092 -rw-------   16384 rw
pgsql    postgres   54086   20 /pgsql   11046093 -rw-------   16384 rw
pgsql    postgres   54086   21 /pgsql   11046110 -rw-------    8192 rw
pgsql    postgres   54086   22 /pgsql   11046156 -rw-------   16384 rw
pgsql    postgres   54086   23 /pgsql   11046248 -rw-------   16384 rw
pgsql    postgres    8056   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres    8056    6 /pgsql   15780331 -rw-------    8192 rw
pgsql    postgres    8056   10 /pgsql   15780295 -rw-------   16384 rw
pgsql    postgres    8056   13 /pgsql   15780365 -rw-------  1712128 rw
pgsql    postgres    8056   15 /pgsql   15780415 -rw-------  1425408 rw
pgsql    postgres    8056   17 -        11053305 -rw-------  16777216 rw
pgsql    postgres    8056   19 /pgsql   15780356 -rw-------  155648 rw
pgsql    postgres    8056   20 /pgsql   15780382 -rw-------  368640 rw
pgsql    postgres    8056   24 /pgsql   11046110 -rw-------    8192 rw
pgsql    postgres    8056   25 /pgsql   11046156 -rw-------   16384 rw
pgsql    postgres    8056   26 /pgsql   11046248 -rw-------   16384 rw
pgsql    postgres   45412   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres   45412   12 /pgsql   15780331 -rw-------    8192 rw
pgsql    postgres   45412   13 /pgsql   15780329 -rw-------   16384 rw
pgsql    postgres   45412   14 /pgsql   15780349 -rw-------   24576 rw
pgsql    postgres   45412   15 /pgsql   15780397 -rw-------   16384 rw
pgsql    postgres   45412   16 /pgsql   15780293 -rw-------   32768 rw
pgsql    postgres   45412   23 /pgsql   15780295 -rw-------   16384 rw
pgsql    postgres   45412   24 /pgsql   15780348 -rw-------    8192 rw
pgsql    postgres   45412   25 /pgsql   15780401 -rw-------   16384 rw
pgsql    postgres   45412   31 /pgsql   15780380 -rw-------   81920 rw
pgsql    postgres   45412   37 -        11047405 -rw-------  16777216 rw
pgsql    postgres   45412   39 /pgsql   15780405 -rw-------   16384 rw
pgsql    postgres   45412   40 /pgsql   15780351 -rw-------  131072 rw
pgsql    postgres   45412   41 /pgsql   15780403 -rw-------   16384 rw
pgsql    postgres   45412   42 /pgsql   15780296 -rw-------  458752 rw
pgsql    postgres   45412   43 /pgsql   15780396 -rw-------   40960 rw
pgsql    postgres   45412   44 /pgsql   15780337 -rw-------  114688 rw
pgsql    postgres   45412   45 /pgsql   15780307 -rw-------   32768 rw
pgsql    postgres   45412   46 /pgsql   11046110 -rw-------    8192 rw
pgsql    postgres   45412   47 /pgsql   11046156 -rw-------   16384 rw
pgsql    postgres   45412   48 /pgsql   11046248 -rw-------   16384 rw
pgsql    postgres   45412   49 /pgsql   15780399 -rw-------   32768 rw
pgsql    postgres   45412   50 /pgsql   15780350 -rw-------   32768 rw
pgsql    postgres   45412   53 /pgsql   15780292 -rw-------   32768 rw
pgsql    postgres   45412   54 /pgsql   15780371 -rw-------   16384 rw
pgsql    postgres   45412   55 /pgsql   15780339 -rw-------   16384 rw
pgsql    postgres   45401   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres   45400   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres   45400    8 /pgsql   11046614 -rw-------    8192 rw
pgsql    postgres   45399   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres   45398   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres   45398    8 -        11053293 -rw-------  16777216 rw
pgsql    postgres   45397   wd /pgsql   11045888 drwx------     512  r
pgsql    postgres   45397    6 /pgsql   11053304 -rw-------  16777216 rw
pgsql    postgres   45395   wd /pgsql   11045888 drwx------     512  r

Script done on Wed May 29 18:54:13 2013

[-- Attachment #3 --]
Script started on Wed May 29 18:57:01 2013
./diskspacecheck.sh
Filesystem     Size    Used   Avail Capacity iused ifree %iused  Mounted on
/dev/da2s1d    128G    101G     17G    86%     27k   17M    0%   /pgsql
ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP

UMA Kegs:               208,      0,      86,      16,      86,   0,   0
UMA Zones:             1408,      0,      86,       0,      86,   0,   0
UMA Slabs:              568,      0,    7019,     324,  609264,   0,   0
UMA RCntSlabs:          568,      0,    4335,     852,  387257,   0,   0
UMA Hash:               256,      0,       0,       0,       3,   0,   0
16 Bucket:              152,      0,      25,      75,     209,   0,   0
32 Bucket:              280,      0,      47,     149,     339,   6,   0
64 Bucket:              536,      0,      78,      76,     752,  90,   0
128 Bucket:            1048,      0,     794,     223, 1511916,14953566,   0
VM OBJECT:              232,      0,   79741,  106291,137621271,   0,   0
MAP:                    232,      0,       7,      25,       7,   0,   0
KMAP ENTRY:             120, 1128710,      78,   51134,1725903331,   0,   0
MAP ENTRY:              120,      0,     582,    7385,293021417,   0,   0
fakepg:                 120,      0,       0,       0,       0,   0,   0
mt_zone:               4112,      0,     326,       1,     326,   0,   0
16:                      16,      0,    2813,    1555,14747990,   0,   0
32:                      32,      0,    2944,    1399,48861396,   0,   0
64:                      64,      0,    5369,    3311,80611779,   0,   0
128:                    128,      0,    9844,    4917,667749833,   0,   0
256:                    256,      0,     568,    5657,2814867773,   0,   0
512:                    512,      0,    2758,    2030,17565631,   0,   0
1024:                  1024,      0,      53,    1227, 2136674,   0,   0
2048:                  2048,      0,    5628,    1236, 4009250,   0,   0
4096:                  4096,      0,     418,    1054, 5175751,   0,   0
Files:                   80,      0,      82,   11123,193060836,   0,   0
TURNSTILE:              136,      0,    1939,     121,    1948,   0,   0
umtx pi:                 96,      0,       0,       0,       0,   0,   0
MAC labels:              40,      0,       0,       0,       0,   0,   0
PROC:                  1184,      0,      43,    1442, 4406995,   0,   0
THREAD:                1128,      0,    1549,     389,    1951,   0,   0
SLEEPQUEUE:              80,      0,    1939,     265,    1948,   0,   0
VMSPACE:                392,      0,      24,    1566, 4406978,   0,   0
cpuset:                  72,      0,      85,      65,      85,   0,   0
audit_record:           960,      0,       0,       0,       0,   0,   0
mbuf_packet:            256,      0,    4080,    1296,688503958,   0,   0
mbuf:                   256,      0,    1021,    1973,1942028265,   0,   0
mbuf_cluster:          2048,  25600,    5376,    1894, 1693184,   0,   0
mbuf_jumbo_page:       4096,  12800,       0,     700,83316546,   0,   0
mbuf_jumbo_9k:         9216,   6400,       0,       0,       0,   0,   0
mbuf_jumbo_16k:       16384,   3200,       0,       0,       0,   0,   0
mbuf_ext_refcnt:          4,      0,       0,       0,       0,   0,   0
g_bio:                  232,      0,       0,    6064,2877091303,   0,   0
ttyinq:                 160,      0,     135,     393,    2505,   0,   0
ttyoutq:                256,      0,      72,     333,    1336,   0,   0
ata_request:            328,      0,       0,      84,      32,   0,   0
ata_composite:          336,      0,       0,       0,       0,   0,   0
VNODE:                  480,      0,  224470,   34810,153776557,   0,   0
VNODEPOLL:              112,      0,       0,     132,       6,   0,   0
NAMEI:                 1024,      0,       0,    1212,418912670,   0,   0
S VFS Cache:            108,      0,  231965,   44113,158049530,   0,   0
STS VFS Cache:          148,      0,       0,       0,       0,   0,   0
L VFS Cache:            328,      0,     681,    4959, 7376729,   0,   0
LTS VFS Cache:          368,      0,       0,       0,       0,   0,   0
NCLNODE:                568,      0,       0,       0,       0,   0,   0
DIRHASH:               1024,      0,    1978,     606, 2981880,   0,   0
Mountpoints:            792,      0,       8,      22,       8,   0,   0
pipe:                   728,      0,       2,    1228, 3848943,   0,   0
ksiginfo:               112,      0,    1474,    1430, 9770535,   0,   0
itimer:                 344,      0,       1,      21,       1,   0,   0
KNOTE:                  128,      0,       0,    1711, 1098021,   0,   0
socket:                 680, 100002,      26,    1432, 9367941,   0,   0
ipq:                     56,    819,       0,       0,       0,   0,   0
udp_inpcb:              392, 100000,      12,    1418, 7328293,   0,   0
udpcb:                   16, 100128,      12,    1836, 7328293,   0,   0
tcp_inpcb:              392, 100000,       5,    1755,  652060,   0,   0
tcpcb:                  976, 100000,       5,    1767,  652060,   0,   0
tcptw:                   72,  20000,       0,     550,    3254,   0,   0
syncache:               152,  15375,       0,     625,  505439,   0,   0
hostcache:              136,  15372,       4,     444,    1820,   0,   0
tcpreass:                40,   1680,       0,    1596,   21690,   0,   0
sackhole:                32,      0,       0,     909,     875,   0,   0
sctp_ep:               1376, 100000,       0,       0,       0,   0,   0
sctp_asoc:             2288,  40000,       0,       0,       0,   0,   0
sctp_laddr:              48,  80064,       0,     288,       6,   0,   0
sctp_raddr:             704,  80000,       0,       0,       0,   0,   0
sctp_chunk:             136, 400008,       0,       0,       0,   0,   0
sctp_readq:             104, 400032,       0,       0,       0,   0,   0
sctp_stream_msg_out:    112, 400026,       0,       0,       0,   0,   0
sctp_asconf:             40, 400008,       0,       0,       0,   0,   0
sctp_asconf_ack:         48, 400032,       0,       0,       0,   0,   0
ripcb:                  392, 100000,       0,     100,     122,   0,   0
unpcb:                  240, 100000,       8,    1320, 1387460,   0,   0
rtentry:                200,      0,      23,      91,      23,   0,   0
selfd:                   56,      0,    2015,    1324,300188200,   0,   0
SWAPMETA:               288, 116519,      33,     565, 1053574,   0,   0
FFS inode:              168,      0,  224434,   41568,153774322,   0,   0
FFS1 dinode:            128,      0,       0,       0,       0,   0,   0
FFS2 dinode:            256,      0,  224434,   40271,153774316,   0,   0

         Type InUse MemUse HighUse Requests  Size(s)
CAM dev queue     4     1K       -        4  128
  md_sii_data     0     0K       -       38  512
      CAM XPT   571  1045K       -      943  16,32,64,128,256,1024,2048
       isadev     8     1K       -        8  128
         UART     6     4K       -        6  16,512,1024
     acpiintr     1     1K       -        1  64
         cdev     7     2K       -        7  256
       acpica  1451   139K       -  6308604  16,32,64,128,256,512,1024
        sigio     1     1K       -        1  64
     filedesc    46    24K       -  4904058  16,32,64,128,256,512,1024,2048,4096
         kenv    78    11K       -       89  16,32,64,128
       kqueue     0     0K       -  1056987  256,512,2048
    proc-args    24     2K       -  6967442  16,32,64,128,256
        hhook     2     1K       -        2  128
     acpitask     1     2K       -        1  2048
      ithread    99    16K       -       99  32,128,256
    CAM queue    19     7K       -      407  16,32,64,128,256,512,2048
       KTRACE   100    13K       -      100  128
       linker   158    11K       -      160  16,32,64,512
        lockf    20     3K       -    24008  64,128
   loginclass     2     1K       -    59065  64
       ip6ndp    12     1K       -       14  64,128
         temp    51     2K       -  8220158  16,32,64,128,256,512,1024,2048,4096
       devbuf  8648 41889K       -    10143  16,32,64,128,256,512,1024,2048,4096
       module   477    60K       -      477  128
    ciss_data    15    14K       -     5921  16,128,256,512,1024,2048,4096
     mtx_pool     2    16K       -        2  
       USBdev    34     8K       -       44  64,128,512,4096
          USB    52    31K       -       66  16,32,64,128,256,2048,4096
     pmchooks     1     1K       -        1  128
      subproc  1531   920K       -  4408484  512,4096
         proc     2    16K       -        2  
      session    21     3K       -  3550683  128
         pgrp    23     3K       -  3555127  128
         cred    42     7K       - 32877162  64,256
      uidinfo     4     3K       -   186861  128,2048
       plimit    14     4K       -   770763  256
      ata_pci     1     1K       -        1  64
      scsi_cd     0     0K       -       10  16
    sysctltmp     0     0K       -    14241  16,32,64,128,4096
    sysctloid  4978   251K       -     5360  16,32,64,128
       sysctl     0     0K       -  7956012  16,32,64
      tidhash     1    16K       -        1  
      callout     7 14336K       -        7  
         umtx  3876   485K       -     3894  128
     p1003.1b     1     1K       -        1  16
         SWAP     2   549K       -        2  64
       bus-sc   113   781K       -     5234  16,32,64,128,256,512,1024,2048,4096
          bus  1282   109K       -     8634  16,32,64,128,256,1024
      devstat    12    25K       -       12  32,4096
 eventhandler    77     6K       -       77  64,128
      acpisem    19     3K       -       19  128
         kobj   330  1320K       -      626  4096
      CAM SIM     5     2K       -        5  256
      Per-cpu     1     1K       -        1  32
   CAM periph    10     3K       -       54  16,32,64,128,256
         rman   248    26K       -      613  16,32,128
         sbuf     1     1K       -     2070  16,32,64,128,256,512,1024,2048,4096
      entropy  1024    64K       -     1024  64
       ctlmem  5062 10113K       -     5062  128,2048
        stack     0     0K       -        2  256
    taskqueue    15     2K       -       15  16,32,128
       Unitno    14     1K       -   596744  32,64
          iov     0     0K       -  1560794  16,32,64,128,256,512
       select  1455   182K       -     1455  128
     ioctlops     0     0K       -  8497618  16,32,64,128,256,512,1024,2048,4096
          msg     4    30K       -        4  2048,4096
          sem     4   196K       -        4  
          shm     1    20K       -  3349283  2048
          tty    22    22K       -      149  1024,2048
          pts     1     1K       -      126  256
     mbuf_tag     0     0K       - 43344699  32,128
        shmfd     1     8K       -        1  
          pcb    20   157K       -   629089  16,32,128,1024,2048,4096
       soname     3     1K       - 52230049  16,32,128
          acl     0     0K       -   144101  4096
       biobuf     0     0K       -    14683  2048
     vfscache     1  8192K       -        1  
   cl_savebuf     0     0K       - 101708558  64,128
     vfs_hash     1  4096K       -        1  
       DEVFS1   114    57K       -      262  512
       vnodes     2     1K       -        4  64,256
       DEVFS3   130    33K       -      285  256
        mount   106     5K       -      176  16,32,64,128,256
  vnodemarker     0     0K       -  5062382  512
          BPF     9     2K       -        9  128
  ether_multi    56     3K       -       66  16,32,64
       ifaddr    87    22K       -       87  32,64,128,256,512,4096
        ifnet    10    19K       -       10  128,2048
        clone     6    24K       -        6  4096
       arpcom     2     1K       -        2  16
      lltable    31    13K       -    58482  256,512
        DEVFS    17     1K       -       20  16,128
       DEVFSP     1     1K       -        2  64
     routetbl    44     6K       -  1464226  32,64,128,256,512
         igmp     9     3K       -        9  256
     in_multi     3     1K       -        3  256
    sctp_iter     0     0K       -        4  256
     sctp_ifn     3     1K       -        3  128
     sctp_ifa     7     1K       -        7  128
     sctp_vrf     1     1K       -        1  64
    sctp_a_it     0     0K       -        4  16
    hostcache     1    28K       -        1  
     syncache     1    96K       -        1  
    in6_multi    28     4K       -       28  32,256
          mld     9     2K       -        9  128
          rpc     2     1K       -        2  256
audit_evclass   179     6K       -      218  32
     savedino     0     0K       -  3083227  256
     freework     1     1K       - 80548874  16,128
    newdirblk     0     0K       -    18337  64
       dirrem     0     0K       - 13083035  128
        mkdir     0     0K       -    29436  128
       diradd     0     0K       - 13091263  128
     freefile     0     0K       -  6713952  64
     freeblks     0     0K       -  6611315  128
     freefrag     0     0K       - 427582922  128
     indirdep     0     0K       -  6251917  128
       newblk     1   512K       - 2781867289  256
    bmsafemap     1     8K       -  8938393  256
     inodedep     1  4096K       -  7832104  512
      pagedep     1   512K       -   139126  256
  ufs_dirhash  1467   373K       -   401649  16,32,64,128,256,512,1024
    ufs_mount    18    69K       -       18  512,2048,4096
    vm_pgdata     2   129K       -        2  128
      UMAHash     3    22K       -       13  512,1024,2048,4096
      memdesc     1     4K       -        1  4096
     atkbddev     2     1K       -        2  64
    pfs_nodes    21     6K       -       21  256
  pfs_vncache     1     1K       -      260  64
       ctlblk   200  1600K       -      200  
         GEOM   127    40K       -     1312  16,32,64,128,256,512,1024,2048
      ramdisk     1  4096K       -        1  
      acpidev    30     2K       -       30  64
      ctlpool   532   142K       -      532  32,512
       kbdmux     7    18K       -        8  16,512,1024,2048
       apmdev     1     1K       -        1  128
   madt_table     0     0K       -        1  4096
       feeder     7     1K       -        7  32
      scsi_da     0     0K       -      266  16
     pci_link    16     2K       -       16  16,128
    raid_data     0     0K       -      234  32,128,256
      io_apic     1     2K       -        1  2048
          MCA     8     1K       -        8  128
          msi     3     1K       -        3  128
     nexusdev     3     1K       -        3  16
md_nvidia_data     0     0K       -       38  512
debug.acpi.suspend_bounce: 0
debug.acpi.reset_clock: 1
debug.acpi.interpreter_slack: 1
debug.acpi.enable_debug_objects: 0
debug.acpi.acpi_ca_version: 20110527
debug.acpi.cpu_unordered: 0
debug.acpi.ec.timeout: 750
debug.acpi.ec.polled: 0
debug.acpi.ec.burst: 0
debug.acpi.batt.batt_sleep_ms: 0
debug.acpi.resume_beep: 0
debug.firewire_debug: 0
debug.fwmem_debug: 0
debug.if_fwe_debug: 0
debug.if_fwip_debug: 0
debug.ipw: 0
debug.iwi: 0
debug.mddebug: 0
debug.wpi: 0
debug.elf64_legacy_coredump: 0
debug.bootverbose: 0
debug.boothowto: 0
debug.cpufreq.verbose: 0
debug.cpufreq.lowest: 0
debug.fail_point.sysctl_running: off
debug.fail_point.buf_pressure: off
debug.fail_point.nlm_deny_grant: off
debug.adaptive_machine_arch: 1
debug.sizeof.cdev_priv: 376
debug.sizeof.cdev: 288
debug.sizeof.g_bioq: 56
debug.sizeof.g_consumer: 96
debug.sizeof.g_provider: 136
debug.sizeof.g_geom: 160
debug.sizeof.g_class: 160
debug.sizeof.kinfo_proc: 1088
debug.sizeof.buf: 608
debug.sizeof.bio: 232
debug.sizeof.proc: 1184
debug.sizeof.vnode: 480
debug.sizeof.devstat: 288
debug.sizeof.namecache: 72
debug.osd: 0
debug.trace_on_panic: 1
debug.debugger_on_panic: 1
debug.ncores: 5
debug.to_avg_mpcalls: 755
debug.to_avg_lockcalls: 303
debug.to_avg_gcalls: 514
debug.to_avg_depth: 2033
debug.umtx.umtx_pi_allocated: 0
debug.clocktime: 0
debug.kdb.alt_break_to_debugger: 0
debug.kdb.break_to_debugger: 0
debug.kdb.trap_code: 0
debug.kdb.trap: 0
debug.kdb.panic: 0
debug.kdb.enter: 0
debug.kdb.current: 
debug.rman_debug: 0
debug.iosize_max_clamp: 1
debug.disablefullpath: 0
debug.disablecwd: 0
debug.vfscache: 1
debug.numcachehv: 25189
debug.numcache: 232646
debug.numneg: 12188
debug.nchash: 1048575
debug.vnlru_nowhere: 0
debug.rush_requests: 370281
debug.if_tun_debug: 0
debug.nlm_debug: 0
debug.fsckcmds: 0
debug.collectsnapstats: 0
debug.snapdebug: 0
debug.dopersistence: 0
debug.softdep.cleanup_failures: 369634
debug.softdep.cleanup_retries: 826
debug.softdep.cleanup_high_delay: 14
debug.softdep.cleanup_inorequests: 0
debug.softdep.cleanup_blkrequests: 369690
debug.softdep.jwait_newblk: 0
debug.softdep.jwait_inode: 0
debug.softdep.jwait_freeblks: 0
debug.softdep.jwait_filepage: 0
debug.softdep.journal_wait: 0
debug.softdep.journal_min: 0
debug.softdep.journal_low: 0
debug.softdep.jnewblk_rollback: 0
debug.softdep.jaddref_rollback: 0
debug.softdep.dir_entry: 195660
debug.softdep.direct_blk_ptrs: 339023
debug.softdep.inode_bitmap: 2503734
debug.softdep.indir_blk_ptrs: 26881
debug.softdep.sync_limit_hit: 591
debug.softdep.ino_limit_hit: 0
debug.softdep.blk_limit_hit: 369220
debug.softdep.ino_limit_push: 0
debug.softdep.blk_limit_push: 369690
debug.softdep.worklist_push: 2156
debug.softdep.maxindirdeps: 50
debug.softdep.tickdelay: 2
debug.softdep.max_softdeps: 2352308
debug.softdep.write.jfsync: 0
debug.softdep.write.jtrunc: 0
debug.softdep.write.sbdep: 0
debug.softdep.write.jsegdep: 0
debug.softdep.write.jseg: 0
debug.softdep.write.jfreefrag: 0
debug.softdep.write.jfreeblk: 0
debug.softdep.write.jnewblk: 0
debug.softdep.write.jmvref: 0
debug.softdep.write.jremref: 0
debug.softdep.write.jaddref: 0
debug.softdep.write.freedep: 0
debug.softdep.write.freework: 0
debug.softdep.write.newdirblk: 0
debug.softdep.write.dirrem: 0
debug.softdep.write.mkdir: 0
debug.softdep.write.diradd: 0
debug.softdep.write.freefile: 0
debug.softdep.write.freeblks: 668969
debug.softdep.write.freefrag: 0
debug.softdep.write.allocindir: 2248015621
debug.softdep.write.indirdep: 115440
debug.softdep.write.allocdirect: 74254149
debug.softdep.write.newblk: 0
debug.softdep.write.bmsafemap: 2393944
debug.softdep.write.inodedep: 6801164
debug.softdep.write.pagedep: 320322
debug.softdep.current.jfsync: 0
debug.softdep.current.jtrunc: 0
debug.softdep.current.sbdep: 0
debug.softdep.current.jsegdep: 0
debug.softdep.current.jseg: 0
debug.softdep.current.jfreefrag: 0
debug.softdep.current.jfreeblk: 0
debug.softdep.current.jnewblk: 0
debug.softdep.current.jmvref: 0
debug.softdep.current.jremref: 0
debug.softdep.current.jaddref: 0
debug.softdep.current.freedep: 0
debug.softdep.current.freework: 0
debug.softdep.current.newdirblk: 0
debug.softdep.current.dirrem: 0
debug.softdep.current.mkdir: 0
debug.softdep.current.diradd: 0
debug.softdep.current.freefile: 0
debug.softdep.current.freeblks: 0
debug.softdep.current.freefrag: 0
debug.softdep.current.allocindir: 0
debug.softdep.current.indirdep: 0
debug.softdep.current.allocdirect: 0
debug.softdep.current.newblk: 0
debug.softdep.current.bmsafemap: 0
debug.softdep.current.inodedep: 0
debug.softdep.current.pagedep: 0
debug.softdep.total.jfsync: 0
debug.softdep.total.jtrunc: 0
debug.softdep.total.sbdep: 0
debug.softdep.total.jsegdep: 0
debug.softdep.total.jseg: 0
debug.softdep.total.jfreefrag: 0
debug.softdep.total.jfreeblk: 0
debug.softdep.total.jnewblk: 0
debug.softdep.total.jmvref: 0
debug.softdep.total.jremref: 0
debug.softdep.total.jaddref: 0
debug.softdep.total.freedep: 0
debug.softdep.total.freework: 80548873
debug.softdep.total.newdirblk: 18337
debug.softdep.total.dirrem: 13083035
debug.softdep.total.mkdir: 29436
debug.softdep.total.diradd: 13091263
debug.softdep.total.freefile: 6713952
debug.softdep.total.freeblks: 6611315
debug.softdep.total.freefrag: 427582922
debug.softdep.total.allocindir: 0
debug.softdep.total.indirdep: 6225036
debug.softdep.total.allocdirect: 0
debug.softdep.total.newblk: 2781867288
debug.softdep.total.bmsafemap: 8938392
debug.softdep.total.inodedep: 7832103
debug.softdep.total.pagedep: 139125
debug.dobkgrdwrite: 1
debug.bigcgs: 0
debug.dircheck: 0
debug.psm.pkterrthresh: 2
debug.psm.usecs: 500000
debug.psm.secs: 0
debug.psm.errusecs: 0
debug.psm.errsecs: 2
debug.psm.hz: 20
debug.psm.loglevel: 0
debug.vesa.shadow_rom: 0
debug.fdc.settle: 0
debug.fdc.spec2: 16
debug.fdc.spec1: 175
debug.fdc.retries: 10
debug.fdc.debugflags: 0
debug.fdc.fifo: 8
debug.elf32_legacy_coredump: 0
debug.x86bios.int: 0
debug.x86bios.call: 0
debug.hwpstate_verbose: 0
debug.minidump: 1
USER     CMD          PID   FD MOUNT      INUM MODE         SZ|DV R/W

Script done on Wed May 29 18:57:01 2013

[-- Attachment #4 --]
Script started on Wed May 29 18:58:17 2013

# umount /pgsql
umount: unmount of /pgsql failed: Device busy
# umount -f fstat | grep pgsql
# lsof | grep pgsql
lsof: WARNING: compiled for FreeBSD release 9.0-RELEASE-p3; this is 9.1-RELEASE.
syslogd    1126   root   19w  VREG              0,111             454973  636092 /var/log/pgsql
# umount -f /pgsql




# 
# 
# 
# 
# ./diskspacecheck.sh sh -x ./diskspacecheck.sh 
+ df -ih /pgsql
Filesystem     Size    Used   Avail Capacity iused ifree %iused  Mounted on
/dev/da0s1a    495M    354M    101M    78%    2.0k   63k    3%   /
+ vmstat -z
ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP

UMA Kegs:               208,      0,      86,      16,      86,   0,   0
UMA Zones:             1408,      0,      86,       0,      86,   0,   0
UMA Slabs:              568,      0,    7018,     325,  609285,   0,   0
UMA RCntSlabs:          568,      0,    4335,     852,  387257,   0,   0
UMA Hash:               256,      0,       0,       0,       3,   0,   0
16 Bucket:              152,      0,      26,      74,     210,   0,   0
32 Bucket:              280,      0,      47,     149,     339,   6,   0
64 Bucket:              536,      0,      78,      76,     752,  90,   0
128 Bucket:            1048,      0,    2100,       0, 1513222,14953566,   0
VM OBJECT:              232,      0,   74433,  111599,137621727,   0,   0
MAP:                    232,      0,       7,      25,       7,   0,   0
KMAP ENTRY:             120, 1128710,      79,   51133,1725903375,   0,   0
MAP ENTRY:              120,      0,     601,    7366,293022516,   0,   0
fakepg:                 120,      0,       0,       0,       0,   0,   0
mt_zone:               4112,      0,     326,       1,     326,   0,   0
16:                      16,      0,    2800,    1568,14748567,   0,   0
32:                      32,      0,    2933,    1410,48861597,   0,   0
64:                      64,      0,    5350,    5178,80616456,   0,   0
128:                    128,      0,    9846,   59290,667808186,   0,   0
256:                    256,      0,     567,    5658,2814868483,   0,   0
512:                    512,      0,    2736,    5321,17574130,   0,   0
1024:                  1024,      0,      53,    1227, 2137845,   0,   0
2048:                  2048,      0,    5627,    1237, 4009522,   0,   0
4096:                  4096,      0,     419,    1053, 5175797,   0,   0
Files:                   80,      0,      83,   11122,193061404,   0,   0
TURNSTILE:              136,      0,    1939,     121,    1948,   0,   0
umtx pi:                 96,      0,       0,       0,       0,   0,   0
MAC labels:              40,      0,       0,       0,       0,   0,   0
PROC:                  1184,      0,      44,    1441, 4407017,   0,   0
THREAD:                1128,      0,    1549,     389,    1951,   0,   0
SLEEPQUEUE:              80,      0,    1939,     265,    1948,   0,   0
VMSPACE:                392,      0,      25,    1565, 4407000,   0,   0
cpuset:                  72,      0,      85,      65,      85,   0,   0
audit_record:           960,      0,       0,       0,       0,   0,   0
mbuf_packet:            256,      0,    4080,    1296,688503966,   0,   0
mbuf:                   256,      0,    1021,    1973,1942028505,   0,   0
mbuf_cluster:          2048,  25600,    5376,    1894, 1693184,   0,   0
mbuf_jumbo_page:       4096,  12800,       0,     700,83316546,   0,   0
mbuf_jumbo_9k:         9216,   6400,       0,       0,       0,   0,   0
mbuf_jumbo_16k:       16384,   3200,       0,       0,       0,   0,   0
mbuf_ext_refcnt:          4,      0,       0,       0,       0,   0,   0
g_bio:                  232,      0,       0,    6064,2877112412,   0,   0
ttyinq:                 160,      0,     135,     393,    2520,   0,   0
ttyoutq:                256,      0,      72,     333,    1344,   0,   0
ata_request:            328,      0,       0,      84,      32,   0,   0
ata_composite:          336,      0,       0,       0,       0,   0,   0
VNODE:                  480,      0,  201214,   58066,153776569,   0,   0
VNODEPOLL:              112,      0,       0,     132,       6,   0,   0
NAMEI:                 1024,      0,       0,    1212,418913966,   0,   0
S VFS Cache:            108,      0,  208213,   67865,158049543,   0,   0
STS VFS Cache:          148,      0,       0,       0,       0,   0,   0
L VFS Cache:            328,      0,     681,    4959, 7376729,   0,   0
LTS VFS Cache:          368,      0,       0,       0,       0,   0,   0
NCLNODE:                568,      0,       0,       0,       0,   0,   0
DIRHASH:               1024,      0,    1745,     839, 2981880,   0,   0
Mountpoints:            792,      0,       7,      23,       8,   0,   0
pipe:                   728,      0,       2,    1228, 3848951,   0,   0
ksiginfo:               112,      0,    1474,    1430, 9770541,   0,   0
itimer:                 344,      0,       1,      21,       1,   0,   0
KNOTE:                  128,      0,       0,    1711, 1098029,   0,   0
socket:                 680, 100002,      26,    1432, 9368026,   0,   0
ipq:                     56,    819,       0,       0,       0,   0,   0
udp_inpcb:              392, 100000,      12,    1418, 7328378,   0,   0
udpcb:                   16, 100128,      12,    1836, 7328378,   0,   0
tcp_inpcb:              392, 100000,       5,    1755,  652060,   0,   0
tcpcb:                  976, 100000,       5,    1767,  652060,   0,   0
tcptw:                   72,  20000,       0,     550,    3254,   0,   0
syncache:               152,  15375,       0,     625,  505439,   0,   0
hostcache:              136,  15372,       4,     444,    1820,   0,   0
tcpreass:                40,   1680,       0,    1596,   21690,   0,   0
sackhole:                32,      0,       0,     909,     875,   0,   0
sctp_ep:               1376, 100000,       0,       0,       0,   0,   0
sctp_asoc:             2288,  40000,       0,       0,       0,   0,   0
sctp_laddr:              48,  80064,       0,     288,       6,   0,   0
sctp_raddr:             704,  80000,       0,       0,       0,   0,   0
sctp_chunk:             136, 400008,       0,       0,       0,   0,   0
sctp_readq:             104, 400032,       0,       0,       0,   0,   0
sctp_stream_msg_out:    112, 400026,       0,       0,       0,   0,   0
sctp_asconf:             40, 400008,       0,       0,       0,   0,   0
sctp_asconf_ack:         48, 400032,       0,       0,       0,   0,   0
ripcb:                  392, 100000,       0,     100,     122,   0,   0
unpcb:                  240, 100000,       8,    1320, 1387460,   0,   0
rtentry:                200,      0,      23,      91,      23,   0,   0
selfd:                   56,      0,    2014,    1325,300191186,   0,   0
SWAPMETA:               288, 116519,      33,     565, 1053574,   0,   0
FFS inode:              168,      0,  197020,   68982,153774326,   0,   0
FFS1 dinode:            128,      0,       0,       0,       0,   0,   0
FFS2 dinode:            256,      0,  197020,   67685,153774320,   0,   0

+ vmstat -m
         Type InUse MemUse HighUse Requests  Size(s)
CAM dev queue     4     1K       -        4  128
  md_sii_data     0     0K       -       42  512
      CAM XPT   571  1045K       -      948  16,32,64,128,256,1024,2048
       isadev     8     1K       -        8  128
         UART     6     4K       -        6  16,512,1024
     acpiintr     1     1K       -        1  64
         cdev     7     2K       -        7  256
       acpica  1451   139K       -  6308688  16,32,64,128,256,512,1024
        sigio     1     1K       -        1  64
     filedesc    47    25K       -  4904080  16,32,64,128,256,512,1024,2048,4096
         kenv    78    11K       -       89  16,32,64,128
       kqueue     0     0K       -  1056995  256,512,2048
    proc-args    25     2K       -  6967491  16,32,64,128,256
        hhook     2     1K       -        2  128
     acpitask     1     2K       -        1  2048
      ithread    99    16K       -       99  32,128,256
    CAM queue    19     7K       -      407  16,32,64,128,256,512,2048
       KTRACE   100    13K       -      100  128
       linker   158    11K       -      160  16,32,64,512
        lockf    20     3K       -    24008  64,128
   loginclass     2     1K       -    59065  64
       ip6ndp    12     1K       -       14  64,128
         temp    51     2K       -  8221621  16,32,64,128,256,512,1024,2048,4096
       devbuf  8648 41889K       -    10143  16,32,64,128,256,512,1024,2048,4096
       module   477    60K       -      477  128
    ciss_data    15    14K       -     5921  16,128,256,512,1024,2048,4096
     mtx_pool     2    16K       -        2  
       USBdev    34     8K       -       44  64,128,512,4096
          USB    52    31K       -       66  16,32,64,128,256,2048,4096
     pmchooks     1     1K       -        1  128
      subproc  1532   924K       -  4408506  512,4096
         proc     2    16K       -        2  
      session    21     3K       -  3550684  128
         pgrp    24     3K       -  3555157  128
         cred    40     7K       - 32877632  64,256
      uidinfo     4     3K       -   186861  128,2048
       plimit    14     4K       -   770763  256
      ata_pci     1     1K       -        1  64
      scsi_cd     0     0K       -       10  16
    sysctltmp     0     0K       -    14257  16,32,64,128,4096
    sysctloid  4978   251K       -     5360  16,32,64,128
       sysctl     0     0K       -  7956342  16,32,64
      tidhash     1    16K       -        1  
      callout     7 14336K       -        7  
         umtx  3876   485K       -     3894  128
     p1003.1b     1     1K       -        1  16
         SWAP     2   549K       -        2  64
       bus-sc   113   781K       -     5234  16,32,64,128,256,512,1024,2048,4096
          bus  1282   109K       -     8640  16,32,64,128,256,1024
      devstat    12    25K       -       12  32,4096
 eventhandler    77     6K       -       77  64,128
      acpisem    19     3K       -       19  128
         kobj   330  1320K       -      650  4096
      CAM SIM     5     2K       -        5  256
      Per-cpu     1     1K       -        1  32
   CAM periph    10     3K       -       54  16,32,64,128,256
         rman   248    26K       -      613  16,32,128
         sbuf     1     1K       -     2110  16,32,64,128,256,512,1024,2048,4096
      entropy  1024    64K       -     1024  64
       ctlmem  5062 10113K       -     5062  128,2048
        stack     0     0K       -        2  256
    taskqueue    15     2K       -       15  16,32,128
       Unitno    14     1K       -   596766  32,64
          iov     0     0K       -  1560794  16,32,64,128,256,512
       select  1455   182K       -     1455  128
     ioctlops     0     0K       -  8497852  16,32,64,128,256,512,1024,2048,4096
          msg     4    30K       -        4  2048,4096
          sem     4   196K       -        4  
          shm     1    20K       -  3349283  2048
          tty    22    22K       -      150  1024,2048
          pts     1     1K       -      127  256
     mbuf_tag     0     0K       - 43344699  32,128
        shmfd     1     8K       -        1  
          pcb    20   157K       -   629093  16,32,128,1024,2048,4096
       soname     3     1K       - 52230357  16,32,128
          acl     0     0K       -   144101  4096
       biobuf     0     0K       -    14683  2048
     vfscache     1  8192K       -        1  
   cl_savebuf     0     0K       - 101708558  64,128
     vfs_hash     1  4096K       -        1  
       DEVFS1   115    58K       -      264  512
       vnodes     2     1K       -        4  64,256
       DEVFS3   134    34K       -      293  256
        mount    91     5K       -      176  16,32,64,128,256
  vnodemarker     0     0K       -  5062496  512
          BPF     9     2K       -        9  128
  ether_multi    56     3K       -       66  16,32,64
       ifaddr    87    22K       -       87  32,64,128,256,512,4096
        ifnet    10    19K       -       10  128,2048
        clone     6    24K       -        6  4096
       arpcom     2     1K       -        2  16
      lltable    31    13K       -    58482  256,512
        DEVFS    17     1K       -       20  16,128
       DEVFSP     1     1K       -        2  64
     routetbl    44     6K       -  1464244  32,64,128,256,512
         igmp     9     3K       -        9  256
     in_multi     3     1K       -        3  256
    sctp_iter     0     0K       -        4  256
     sctp_ifn     3     1K       -        3  128
     sctp_ifa     7     1K       -        7  128
     sctp_vrf     1     1K       -        1  64
    sctp_a_it     0     0K       -        4  16
    hostcache     1    28K       -        1  
     syncache     1    96K       -        1  
    in6_multi    28     4K       -       28  32,256
          mld     9     2K       -        9  128
          rpc     2     1K       -        2  256
audit_evclass   179     6K       -      218  32
     savedino     0     0K       -  3083227  256
     freework     1     1K       - 80602902  16,128
    newdirblk     0     0K       -    18337  64
       dirrem     0     0K       - 13083035  128
        mkdir     0     0K       -    29436  128
       diradd     0     0K       - 13091263  128
     freefile     0     0K       -  6718108  64
     freeblks     0     0K       -  6615471  128
     freefrag     0     0K       - 427582922  128
     indirdep     0     0K       -  6251917  128
       newblk     1   512K       - 2781867289  256
    bmsafemap     1     8K       -  8938817  256
     inodedep     1  4096K       -  7840416  512
      pagedep     1   512K       -   139126  256
  ufs_dirhash  1407   359K       -   401649  16,32,64,128,256,512,1024
    ufs_mount    15    51K       -       18  512,2048,4096
    vm_pgdata     2   129K       -        2  128
      UMAHash     3    22K       -       13  512,1024,2048,4096
      memdesc     1     4K       -        1  4096
     atkbddev     2     1K       -        2  64
    pfs_nodes    21     6K       -       21  256
  pfs_vncache     1     1K       -      260  64
       ctlblk   200  1600K       -      200  
         GEOM   133    41K       -     1419  16,32,64,128,256,512,1024,2048
      ramdisk     1  4096K       -        1  
      acpidev    30     2K       -       30  64
      ctlpool   532   142K       -      532  32,512
       kbdmux     7    18K       -        8  16,512,1024,2048
       apmdev     1     1K       -        1  128
   madt_table     0     0K       -        1  4096
       feeder     7     1K       -        7  32
      scsi_da     0     0K       -      296  16
     pci_link    16     2K       -       16  16,128
    raid_data     0     0K       -      258  32,128,256
      io_apic     1     2K       -        1  2048
          MCA     8     1K       -        8  128
          msi     3     1K       -        3  128
     nexusdev     3     1K       -        3  16
md_nvidia_data     0     0K       -       42  512
+ sysctl debug
debug.acpi.suspend_bounce: 0
debug.acpi.reset_clock: 1
debug.acpi.interpreter_slack: 1
debug.acpi.enable_debug_objects: 0
debug.acpi.acpi_ca_version: 20110527
debug.acpi.cpu_unordered: 0
debug.acpi.ec.timeout: 750
debug.acpi.ec.polled: 0
debug.acpi.ec.burst: 0
debug.acpi.batt.batt_sleep_ms: 0
debug.acpi.resume_beep: 0
debug.firewire_debug: 0
debug.fwmem_debug: 0
debug.if_fwe_debug: 0
debug.if_fwip_debug: 0
debug.ipw: 0
debug.iwi: 0
debug.mddebug: 0
debug.wpi: 0
debug.elf64_legacy_coredump: 0
debug.bootverbose: 0
debug.boothowto: 0
debug.cpufreq.verbose: 0
debug.cpufreq.lowest: 0
debug.fail_point.sysctl_running: off
debug.fail_point.buf_pressure: off
debug.fail_point.nlm_deny_grant: off
debug.adaptive_machine_arch: 1
debug.sizeof.cdev_priv: 376
debug.sizeof.cdev: 288
debug.sizeof.g_bioq: 56
debug.sizeof.g_consumer: 96
debug.sizeof.g_provider: 136
debug.sizeof.g_geom: 160
debug.sizeof.g_class: 160
debug.sizeof.kinfo_proc: 1088
debug.sizeof.buf: 608
debug.sizeof.bio: 232
debug.sizeof.proc: 1184
debug.sizeof.vnode: 480
debug.sizeof.devstat: 288
debug.sizeof.namecache: 72
debug.osd: 0
debug.trace_on_panic: 1
debug.debugger_on_panic: 1
debug.ncores: 5
debug.to_avg_mpcalls: 743
debug.to_avg_lockcalls: 251
debug.to_avg_gcalls: 491
debug.to_avg_depth: 1734
debug.umtx.umtx_pi_allocated: 0
debug.clocktime: 0
debug.kdb.alt_break_to_debugger: 0
debug.kdb.break_to_debugger: 0
debug.kdb.trap_code: 0
debug.kdb.trap: 0
debug.kdb.panic: 0
debug.kdb.enter: 0
debug.kdb.current: 
debug.rman_debug: 0
debug.iosize_max_clamp: 1
debug.disablefullpath: 0
debug.disablecwd: 0
debug.vfscache: 1
debug.numcachehv: 25158
debug.numcache: 208894
debug.numneg: 11694
debug.nchash: 1048575
debug.vnlru_nowhere: 0
debug.rush_requests: 370281
debug.if_tun_debug: 0
debug.nlm_debug: 0
debug.fsckcmds: 0
debug.collectsnapstats: 0
debug.snapdebug: 0
debug.dopersistence: 0
debug.softdep.cleanup_failures: 369634
debug.softdep.cleanup_retries: 826
debug.softdep.cleanup_high_delay: 14
debug.softdep.cleanup_inorequests: 0
debug.softdep.cleanup_blkrequests: 369690
debug.softdep.jwait_newblk: 0
debug.softdep.jwait_inode: 0
debug.softdep.jwait_freeblks: 0
debug.softdep.jwait_filepage: 0
debug.softdep.journal_wait: 0
debug.softdep.journal_min: 0
debug.softdep.journal_low: 0
debug.softdep.jnewblk_rollback: 0
debug.softdep.jaddref_rollback: 0
debug.softdep.dir_entry: 195660
debug.softdep.direct_blk_ptrs: 339023
debug.softdep.inode_bitmap: 2503734
debug.softdep.indir_blk_ptrs: 26881
debug.softdep.sync_limit_hit: 591
debug.softdep.ino_limit_hit: 0
debug.softdep.blk_limit_hit: 369220
debug.softdep.ino_limit_push: 0
debug.softdep.blk_limit_push: 369690
debug.softdep.worklist_push: 2156
debug.softdep.maxindirdeps: 50
debug.softdep.tickdelay: 2
debug.softdep.max_softdeps: 2352308
debug.softdep.write.jfsync: 0
debug.softdep.write.jtrunc: 0
debug.softdep.write.sbdep: 0
debug.softdep.write.jsegdep: 0
debug.softdep.write.jseg: 0
debug.softdep.write.jfreefrag: 0
debug.softdep.write.jfreeblk: 0
debug.softdep.write.jnewblk: 0
debug.softdep.write.jmvref: 0
debug.softdep.write.jremref: 0
debug.softdep.write.jaddref: 0
debug.softdep.write.freedep: 0
debug.softdep.write.freework: 0
debug.softdep.write.newdirblk: 0
debug.softdep.write.dirrem: 0
debug.softdep.write.mkdir: 0
debug.softdep.write.diradd: 0
debug.softdep.write.freefile: 0
debug.softdep.write.freeblks: 673125
debug.softdep.write.freefrag: 0
debug.softdep.write.allocindir: 2248015621
debug.softdep.write.indirdep: 115440
debug.softdep.write.allocdirect: 74254149
debug.softdep.write.newblk: 0
debug.softdep.write.bmsafemap: 2394368
debug.softdep.write.inodedep: 6805320
debug.softdep.write.pagedep: 320322
debug.softdep.current.jfsync: 0
debug.softdep.current.jtrunc: 0
debug.softdep.current.sbdep: 0
debug.softdep.current.jsegdep: 0
debug.softdep.current.jseg: 0
debug.softdep.current.jfreefrag: 0
debug.softdep.current.jfreeblk: 0
debug.softdep.current.jnewblk: 0
debug.softdep.current.jmvref: 0
debug.softdep.current.jremref: 0
debug.softdep.current.jaddref: 0
debug.softdep.current.freedep: 0
debug.softdep.current.freework: 0
debug.softdep.current.newdirblk: 0
debug.softdep.current.dirrem: 0
debug.softdep.current.mkdir: 0
debug.softdep.current.diradd: 0
debug.softdep.current.freefile: 0
debug.softdep.current.freeblks: 0
debug.softdep.current.freefrag: 0
debug.softdep.current.allocindir: 0
debug.softdep.current.indirdep: 0
debug.softdep.current.allocdirect: 0
debug.softdep.current.newblk: 0
debug.softdep.current.bmsafemap: 0
debug.softdep.current.inodedep: 0
debug.softdep.current.pagedep: 0
debug.softdep.total.jfsync: 0
debug.softdep.total.jtrunc: 0
debug.softdep.total.sbdep: 0
debug.softdep.total.jsegdep: 0
debug.softdep.total.jseg: 0
debug.softdep.total.jfreefrag: 0
debug.softdep.total.jfreeblk: 0
debug.softdep.total.jnewblk: 0
debug.softdep.total.jmvref: 0
debug.softdep.total.jremref: 0
debug.softdep.total.jaddref: 0
debug.softdep.total.freedep: 0
debug.softdep.total.freework: 80602901
debug.softdep.total.newdirblk: 18337
debug.softdep.total.dirrem: 13083035
debug.softdep.total.mkdir: 29436
debug.softdep.total.diradd: 13091263
debug.softdep.total.freefile: 6718108
debug.softdep.total.freeblks: 6615471
debug.softdep.total.freefrag: 427582922
debug.softdep.total.allocindir: 0
debug.softdep.total.indirdep: 6225036
debug.softdep.total.allocdirect: 0
debug.softdep.total.newblk: 2781867288
debug.softdep.total.bmsafemap: 8938816
debug.softdep.total.inodedep: 7840415
debug.softdep.total.pagedep: 139125
debug.dobkgrdwrite: 1
debug.bigcgs: 0
debug.dircheck: 0
debug.psm.pkterrthresh: 2
debug.psm.usecs: 500000
debug.psm.secs: 0
debug.psm.errusecs: 0
debug.psm.errsecs: 2
debug.psm.hz: 20
debug.psm.loglevel: 0
debug.vesa.shadow_rom: 0
debug.fdc.settle: 0
debug.fdc.spec2: 16
debug.fdc.spec1: 175
debug.fdc.retries: 10
debug.fdc.debugflags: 0
debug.fdc.fifo: 8
debug.elf32_legacy_coredump: 0
debug.x86bios.int: 0
debug.x86bios.call: 0
debug.hwpstate_verbose: 0
debug.minidump: 1
+ fstat -f /pgsql
USER     CMD          PID   FD MOUNT      INUM MODE         SZ|DV R/W
root     fstat      12974   wd /         32902 drwxr-xr-x    1024  r
root     fstat      12974 root /             2 drwxr-xr-x     512  r
root     fstat      12974    3 /         49784 -rw-------   40960  r
root     sh         12969 text /         32956 -r-xr-xr-x  142984  r
root     sh         12969   wd /         32902 drwxr-xr-x    1024  r
root     sh         12969 root /             2 drwxr-xr-x     512  r
root     sh         12969   10 /         32978 -rwxr-xr-x      74  r
root     sh         12961 text /         32956 -r-xr-xr-x  142984  r
root     sh         12961   wd /         32902 drwxr-xr-x    1024  r
root     sh         12961 root /             2 drwxr-xr-x     512  r
root     script     12960   wd /         32902 drwxr-xr-x    1024  r
root     script     12960 root /             2 drwxr-xr-x     512  r
root     script     12960    3 -         32982 -rw-r--r--   16833  w
root     sh         12942 text /         32956 -r-xr-xr-x  142984  r
root     sh         12942   wd /         32902 drwxr-xr-x    1024  r
root     sh         12942 root /             2 drwxr-xr-x     512  r
root     login      12941   wd /         32902 drwxr-xr-x    1024  r
root     login      12941 root /             2 drwxr-xr-x     512  r
nagios   nrpe2       8410   wd /             2 drwxr-xr-x     512  r
nagios   nrpe2       8410 root /             2 drwxr-xr-x     512  r
root     moused     68974   wd /             2 drwxr-xr-x     512  r
root     moused     68974 root /             2 drwxr-xr-x     512  r
root     getty       1320   wd /             2 drwxr-xr-x     512  r
root     getty       1320 root /             2 drwxr-xr-x     512  r
root     getty       1319   wd /             2 drwxr-xr-x     512  r
root     getty       1319 root /             2 drwxr-xr-x     512  r
root     getty       1318   wd /             2 drwxr-xr-x     512  r
root     getty       1318 root /             2 drwxr-xr-x     512  r
root     getty       1317   wd /             2 drwxr-xr-x     512  r
root     getty       1317 root /             2 drwxr-xr-x     512  r
root     getty       1316   wd /             2 drwxr-xr-x     512  r
root     getty       1316 root /             2 drwxr-xr-x     512  r
root     getty       1315   wd /             2 drwxr-xr-x     512  r
root     getty       1315 root /             2 drwxr-xr-x     512  r
root     getty       1314   wd /             2 drwxr-xr-x     512  r
root     getty       1314 root /             2 drwxr-xr-x     512  r
root     cron        1275 root /             2 drwxr-xr-x     512  r
smmsp    sendmail    1271 root /             2 drwxr-xr-x     512  r
root     sendmail    1268 root /             2 drwxr-xr-x     512  r
root     sshd        1249   wd /             2 drwxr-xr-x     512  r
root     sshd        1249 root /             2 drwxr-xr-x     512  r
root     ntpd        1212   wd /             2 drwxr-xr-x     512  r
root     ntpd        1212 root /             2 drwxr-xr-x     512  r
root     snmpd       1180   wd /             2 drwxr-xr-x     512  r
root     snmpd       1180 root /             2 drwxr-xr-x     512  r
root     syslogd     1126   wd /             2 drwxr-xr-x     512  r
root     syslogd     1126 root /             2 drwxr-xr-x     512  r
root     devd        1012 text /           105 -r-xr-xr-x  454528  r
root     devd        1012   wd /             2 drwxr-xr-x     512  r
root     devd        1012 root /             2 drwxr-xr-x     512  r
root     adjkerntz    122 text /           100 -r-xr-xr-x    9264  r
root     adjkerntz    122   wd /             2 drwxr-xr-x     512  r
root     adjkerntz    122 root /             2 drwxr-xr-x     512  r
root     init           1 text /           112 -r-xr-xr-x  792128  r
root     init           1   wd /             2 drwxr-xr-x     512  r
root     init           1 root /             2 drwxr-xr-x     512  r
root     kernel         0   wd /             2 drwxr-xr-x     512  r
root     kernel         0 root /             2 drwxr-xr-x     512  r
# df -h /pgsql
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/da0s1a    495M    354M    101M    78%    /
# mount /pgsql
# mount /pgsqldf -h
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/da2s1d    128G     82G     36G    70%    /pgsql
# du -sh /pgsql
 82G	/pgsql
# exit

Script done on Wed May 29 19:00:12 2013

Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?51A73076.8020609>