From owner-freebsd-stable@FreeBSD.ORG Tue Nov 27 20:10:56 2012 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B6586C53 for ; Tue, 27 Nov 2012 20:10:56 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 413B28FC0C for ; Tue, 27 Nov 2012 20:10:55 +0000 (UTC) Received: by mail-bk0-f54.google.com with SMTP id je9so5207674bkc.13 for ; Tue, 27 Nov 2012 12:10:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:content-transfer-encoding:date:subject:to :message-id:mime-version:x-mailer; bh=wVu/PnohnykXSZgq4gNiZBWgF0ktmJK5pvuVl4YnbmI=; b=TOWv/dlpDma0ell7GABgWv08SrXTsSx4y7aBqIxtjV6Pz7Jz0QcIrPOqdzS+n7D3VV DP7koYOMu6dk0V1Kqo4IIWbVwSR+zu1p97V1/v/WP4ZzNua8s0jDBHtw/GTd20qxNAVh rm91oyg6UbiqjbbtWnH/LsqEnnnTdilKEUx5Q697r9Zk8SiUGN7GrvDJTVkshMNghj10 4+fZKw3gig/rTnbBqSctG9POAFRB3Sj28PB+H6XzPehFD2Jb3RhBE37zf4NPhVyQ205F VlR10qu6ccglxL7jM/QIiuwjEJTiwwzbg/rFMl2eb51vITIKKKm5EZB9T88SySZs8RxG xpiQ== Received: by 10.204.154.202 with SMTP id p10mr3025688bkw.29.1354047054965; Tue, 27 Nov 2012 12:10:54 -0800 (PST) Received: from imba-brutale-3.totalterror.net ([93.152.184.10]) by mx.google.com with ESMTPS id 18sm2067372bkv.0.2012.11.27.12.10.53 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 27 Nov 2012 12:10:54 -0800 (PST) From: Nikolay Denev Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Tue, 27 Nov 2012 22:10:52 +0200 Subject: ZFS memory management To: "freebsd-stable@freebsd.org" Message-Id: <7A88B836-C985-446C-A992-A295A2474A38@gmail.com> Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) X-Mailer: Apple Mail (2.1499) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Nov 2012 20:10:56 -0000 Hello list, I have the following question : I have several machines with 196G of RAM = that are using RELENG_9 with ZFS, and are running a very memory intensive java = applications - ElasticSearch The machines are without swap configured and have "vm.swap_enabled=3D0" = in /etc/sysctl.conf. The ElasticSearch processes are using mlockall(2) to pin down their = memory (configured at 40G). And at this point I thought that there would be no problems, but from = time to time, when the machine grows it's=20 ARC memory and there are some other running processes like nginx with = passenger and uwsgi the ElasticSearch process would get killed by the kernel OOM killer with reason "no swap = space available" Of course, I've now tuned down arc_max in /boot/loader.conf, but isn't = this supposed to work automatically? Like ZFS releasing some memory when there is a pressure, instead of the OOM = killer going postal? (at the moment when the process was killed the ZFS ARC was 132G). I understand that this might be problematic as AFAIK ZFS releases memory = asynchronously when the arc_reclaim_thread() is run, which might take some time to be scheduled and complete. Cheers, Nikolay