From owner-freebsd-questions@FreeBSD.ORG Wed Mar 20 15:30:08 2013 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 59FA45C2 for ; Wed, 20 Mar 2013 15:30:08 +0000 (UTC) (envelope-from godders@gmail.com) Received: from mail-qc0-x22e.google.com (mail-qc0-x22e.google.com [IPv6:2607:f8b0:400d:c01::22e]) by mx1.freebsd.org (Postfix) with ESMTP id 16E2BEF3 for ; Wed, 20 Mar 2013 15:30:08 +0000 (UTC) Received: by mail-qc0-f174.google.com with SMTP id z24so884575qcq.19 for ; Wed, 20 Mar 2013 08:30:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type; bh=DaaxNGzHRHDm5/E3P7fxOXG1i1AE12Og5+AaFLY8wkY=; b=D+DYDb/p3Scpqb/nEQhuRX6MYoKugGI+rCxAOCIPktscb5SLWs8tHemYbW0P/B89nP yxIDP2q2xb7q0/JMAhTW6hZeEpv2mKoJyqVl3wg2EazJlU/FP3bSsCYIqoQPQtWziFRo P33L7sjdLhCWrMZ4SGpzSCPide6FU6gOx7E6f4cu0Xrs3e5pcnSCam6qGlf8G4+mD9/C NBcyxwOHO7h3+MPzEUohX09HpUwA4a+M21D6/pmUxQ8N4wgBYuSxq7v6DVMrLKj3XXJK +Nmk4rah22HUANGv58V0er9+Kp1LJBAJ8THzuwNJxWw5FowBmQxK2TMngtOSaXdLFA6f T9Og== MIME-Version: 1.0 X-Received: by 10.49.12.143 with SMTP id y15mr7275744qeb.27.1363792998782; Wed, 20 Mar 2013 08:23:18 -0700 (PDT) Received: by 10.49.71.145 with HTTP; Wed, 20 Mar 2013 08:23:18 -0700 (PDT) Date: Wed, 20 Mar 2013 15:23:18 +0000 Message-ID: Subject: "Leaking" disk space From: Dan Thomas To: freebsd-questions@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Mar 2013 15:30:08 -0000 Hi Guys, We're seeing a problem with some of our FreeBSD/PostgreSQL servers "leaking" quite significant amounts of disk space: > df -h /usr/local/pgsql/ Filesystem Size Used Avail Capacity Mounted on /dev/mfid1s1d 1.1T 772G 222G 78% /usr/local/pgsql > du -sh /usr/local/pgsql/ 741G /usr/local/pgsql/ Stopping Postgres doesn't fix it, but rebooting does which points at the OS rather than PG to me. However, the leak is only apparent in the dedicated pgsql partition, and only on our database servers, so PostgreSQL seems to at least be involved. The partition itself is a relatively standard UFS partition: > grep /usr/local/pgsql /etc/fstab /dev/mfid1s1d /usr/local/pgsql ufs rw 2 2 > tunefs -p /usr/local/pgsql/ tunefs: POSIX.1e ACLs: (-a) disabled tunefs: NFSv4 ACLs: (-N) disabled tunefs: MAC multilabel: (-l) disabled tunefs: soft updates: (-n) enabled tunefs: gjournal: (-J) disabled tunefs: trim: (-t) disabled tunefs: maximum blocks per file in a cylinder group: (-e) 2048 tunefs: average file size: (-f) 16384 tunefs: average number of files in a directory: (-s) 64 tunefs: minimum percentage of free space: (-m) 8% tunefs: optimization preference: (-o) time tunefs: volume label: (-L) LSOF isn't showing any open files: > lsof +L /usr/local/pgsql/ | awk '{ print $8 }' | grep 0 | wc -l 0 We're not creating filesystem snapshots: > find /usr/local/pgsql/ -flags snapshot > Not all of our servers are leaking space, it's only the more recently-installed systems. Here's a quick breakdown of versions: FreeBSD PostgreSQL Leaking? 8.0 8.4.4 no 8.2 9.0.4 no 8.3 9.1.4 yes 8.3 9.2.3 yes 9.1 9.2.3 yes Any ideas what's going on here, or where we could start debugging? Thanks, Dan