From owner-freebsd-hackers@FreeBSD.ORG Sun Jan 8 00:05:29 2006 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A0C0F16A41F for ; Sun, 8 Jan 2006 00:05:29 +0000 (GMT) (envelope-from ilyaver@mail.ru) Received: from mx2.mail.ru (mx2.mail.ru [194.67.23.122]) by mx1.FreeBSD.org (Postfix) with ESMTP id 375EE43D45 for ; Sun, 8 Jan 2006 00:05:27 +0000 (GMT) (envelope-from ilyaver@mail.ru) Received: from [85.21.147.160] (port=1424 helo=UA0WONE34JO8YO8) by mx2.mail.ru with esmtp id 1EvO3p-000JR1-00 for freebsd-hackers@freebsd.org; Sun, 08 Jan 2006 03:05:25 +0300 Date: Sun, 8 Jan 2006 03:04:44 +0300 From: Ilya E Veretenkin X-Mailer: The Bat! (v3.0) Professional X-Priority: 3 (Normal) Message-ID: <72482398.20060108030444@mail.ru> To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: FreeBSD 6.0 - amount of total visible memory is decreasing in time X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Ilya E Veretenkin List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jan 2006 00:05:29 -0000 Problem: Total amount of visible memory is decreasing while system is running (total amount = sum of all memory fields in 'top' utility output, in other words: Active+Inact+Wired+Cache+Buf+Free) OS: FreeBSD 6 Server: 2xAmd64bit, 4GB memory, tyan motherboard hw.physmem: 3478589440 hw.realmem: 3488743424 These values don't change in time. Just after server boot, the sum of memory fields in 'top' utility output is 3405M (this is correct: hw.physmem is the same). But this sum(i.e. total visible memory for OS) is decreasing in time, according to top's output. After all, server begins to swap and needs to be rebooted again. Here goes monitoring log. It was created by hand-written perl script, which is called every minute and which summarizes 'top' utility's 6 memory fields. Total memory(Active+Inact+Wired+Cache+Buf+Free) is in the second column in the following log: Sat Jan 7 20:37:00 MSK 2006 3224 Sat Jan 7 20:38:00 MSK 2006 3224 Sat Jan 7 20:39:00 MSK 2006 3224 Sat Jan 7 20:40:00 MSK 2006 3225 Sat Jan 7 20:41:00 MSK 2006 3223 Sat Jan 7 20:42:00 MSK 2006 3224 Sat Jan 7 20:43:00 MSK 2006 3222 Sat Jan 7 20:44:00 MSK 2006 3223 Sat Jan 7 20:45:00 MSK 2006 3221 Sat Jan 7 20:46:00 MSK 2006 3218 Sat Jan 7 20:47:00 MSK 2006 3217 Sat Jan 7 20:48:00 MSK 2006 3218 Sat Jan 7 20:49:00 MSK 2006 3213 Sat Jan 7 20:50:00 MSK 2006 3212 Sat Jan 7 20:51:00 MSK 2006 3212 Sat Jan 7 20:52:00 MSK 2006 3212 Sat Jan 7 20:53:00 MSK 2006 3211 Sat Jan 7 20:54:00 MSK 2006 3211 Sat Jan 7 20:55:00 MSK 2006 3210 Sat Jan 7 20:56:00 MSK 2006 3211 Sat Jan 7 20:57:00 MSK 2006 3209 Sat Jan 7 20:58:00 MSK 2006 3211 Sat Jan 7 20:59:00 MSK 2006 3210 Sat Jan 7 21:00:00 MSK 2006 3209 Sat Jan 7 21:01:00 MSK 2006 3208 Sat Jan 7 21:02:00 MSK 2006 3207 Sat Jan 7 21:03:00 MSK 2006 3205 Sat Jan 7 21:04:00 MSK 2006 3206 Sat Jan 7 21:05:00 MSK 2006 3206 Sat Jan 7 21:06:00 MSK 2006 3204 Sat Jan 7 21:07:00 MSK 2006 3203 Sat Jan 7 21:08:00 MSK 2006 3204 Sat Jan 7 21:09:00 MSK 2006 3202 Sat Jan 7 21:10:00 MSK 2006 3202 Sat Jan 7 21:11:00 MSK 2006 3204 Sat Jan 7 21:12:00 MSK 2006 3203 Sat Jan 7 21:13:00 MSK 2006 3203 Sat Jan 7 21:14:00 MSK 2006 3203 Sat Jan 7 21:15:00 MSK 2006 3202 Sat Jan 7 21:16:00 MSK 2006 3202 Sat Jan 7 21:17:00 MSK 2006 3202 Sat Jan 7 21:18:00 MSK 2006 3201 Sat Jan 7 21:19:00 MSK 2006 3201 Sat Jan 7 21:20:00 MSK 2006 3201 Sat Jan 7 21:21:00 MSK 2006 3202 Sat Jan 7 21:22:00 MSK 2006 3201 Sat Jan 7 21:23:00 MSK 2006 3200 Sat Jan 7 21:24:00 MSK 2006 3203 Sat Jan 7 21:25:00 MSK 2006 3202 Sat Jan 7 21:26:00 MSK 2006 3201 Sat Jan 7 21:27:00 MSK 2006 3203 Sat Jan 7 21:28:00 MSK 2006 3203 Sat Jan 7 21:29:00 MSK 2006 3203 Sat Jan 7 21:30:00 MSK 2006 3202 Sat Jan 7 21:31:00 MSK 2006 3203 Sat Jan 7 21:32:00 MSK 2006 3203 Sat Jan 7 21:33:00 MSK 2006 3202 Sat Jan 7 21:34:00 MSK 2006 3202 Sat Jan 7 21:35:00 MSK 2006 3201 Sat Jan 7 21:36:00 MSK 2006 3201 Sat Jan 7 21:37:00 MSK 2006 3201 Sat Jan 7 21:38:00 MSK 2006 3200 Sat Jan 7 21:39:00 MSK 2006 3201 Sat Jan 7 21:40:00 MSK 2006 3199 Sat Jan 7 21:41:00 MSK 2006 3201 Sat Jan 7 21:42:01 MSK 2006 3201 Sat Jan 7 21:43:00 MSK 2006 3200 Sat Jan 7 21:44:00 MSK 2006 3200 Sat Jan 7 21:45:00 MSK 2006 3200 Sat Jan 7 21:46:00 MSK 2006 3200 Sat Jan 7 21:47:00 MSK 2006 3199 Sat Jan 7 21:48:00 MSK 2006 3199 Sat Jan 7 21:49:00 MSK 2006 3199 Sat Jan 7 21:50:00 MSK 2006 3198.544 Sat Jan 7 21:51:00 MSK 2006 3199 Sat Jan 7 21:52:00 MSK 2006 3199 Sat Jan 7 21:53:00 MSK 2006 3198 Sat Jan 7 21:54:00 MSK 2006 3197 Sat Jan 7 21:55:00 MSK 2006 3198 Sat Jan 7 21:56:00 MSK 2006 3197 Sat Jan 7 21:57:00 MSK 2006 3198 Sat Jan 7 21:58:00 MSK 2006 3197 Sat Jan 7 21:59:00 MSK 2006 3197 Sat Jan 7 22:00:00 MSK 2006 3198 Sat Jan 7 22:01:00 MSK 2006 3197 Sat Jan 7 22:02:00 MSK 2006 3197 Sat Jan 7 22:03:00 MSK 2006 3197 Sat Jan 7 22:04:00 MSK 2006 3196 Sat Jan 7 22:05:00 MSK 2006 3196 Sat Jan 7 22:06:00 MSK 2006 3195 Sat Jan 7 22:07:00 MSK 2006 3195.62 Sat Jan 7 22:08:00 MSK 2006 3196 Sat Jan 7 22:09:00 MSK 2006 3196 Sat Jan 7 22:10:00 MSK 2006 3196 Sat Jan 7 22:11:00 MSK 2006 3194 Sat Jan 7 22:12:00 MSK 2006 3195 Sat Jan 7 22:13:00 MSK 2006 3196 Sat Jan 7 22:14:00 MSK 2006 3195 Sat Jan 7 22:14:00 MSK 2006 3195 (+-2 megabyte fluctuations here is a result of rounding) As you can see, the memory, which was visible from OS's point of view, decreased from 3224 to 3195 Megabytes in less than 2 hours, according to 'top' utility's output. And this process will continue to the point, where swapping begins. Server during this monitoring was running MySql 4.1.15 + Apache 1.3 under heavy load. I'd like to mention one more time: _total_ memory decreases (not free memory). The problem seems very strange for me. How to get rid of such memory loss? And what is the reason of such memory loss?