From owner-freebsd-performance@FreeBSD.ORG Tue Mar 23 12:02:18 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 56AFF16A4FE for ; Tue, 23 Mar 2004 12:02:18 -0800 (PST) Received: from mail.gotfrag.com (unknown [66.208.110.13]) by mx1.FreeBSD.org (Postfix) with ESMTP id DC69C43D5A for ; Tue, 23 Mar 2004 12:02:17 -0800 (PST) (envelope-from jcoene@gotfrag.com) Received: from jcoene2 (roc-66-66-149-175.rochester.rr.com [66.66.149.175]) by mail.gotfrag.com (8.12.10/8.12.8) with ESMTP id i2NK2Gam077177; Tue, 23 Mar 2004 15:02:16 -0500 (EST) (envelope-from jcoene@gotfrag.com) Message-Id: <200403232002.i2NK2Gam077177@mail.gotfrag.com> From: "Jason Coene" To: Date: Tue, 23 Mar 2004 15:02:22 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 Thread-Index: AcQREb4wFaIoOWDESb2QBjF6FBso/g== X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Subject: FreeBSD, PostgreSQL, semwait and sbwait! X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Mar 2004 20:02:18 -0000 Hello all, We're having a substantial problem with our FreeBSD 5.2 database server running PostgreSQL - it's getting a lot of traffic (figure about 3,000 queries per second), but queries are slow, and it's seemingly waiting on other things than CPU time. The database server is a dual P4-2.8 w/ HT enabled (kernel finds 4 processors), 2GB RAM, 4 disk Serial ATA on 3ware RAID, gigabit Ethernet connection to web servers. It's running FreeBSD 5.2 and PostgreSQL 7.4.1. The server is taking a while to respond to both connections, and then queries (between 1-3 seconds, on a query that should execute in 0.05 or less). The CPU usage for the server never goes above 30% (70% idle), and the CPU time that's in use is nearly always split equal between user and system. The system is using Doing a "top", this is what we see: last pid: 51833; load averages: 13.72, 11.74, 10.01 up 0+01:55:45 15:00:03 116 processes: 1 running, 99 sleeping, 16 lock CPU states: 14.6% user, 0.0% nice, 23.7% system, 0.2% interrupt, 61.5% idle Mem: 91M Active, 1043M Inact, 160M Wired, 52K Cache, 112M Buf, 644M Free Swap: 4096M Total, 4096M Free 20354 pgsql 131 0 80728K 5352K select 0 0:24 1.71% 1.71% postgres 36415 pgsql 4 0 81656K 67468K sbwait 2 0:00 3.23% 0.59% postgres 36442 pgsql 128 0 82360K 15868K select 2 0:00 1.75% 0.24% postgres 36447 pgsql -4 0 82544K 10616K semwai 0 0:00 2.05% 0.20% postgres 36461 pgsql -4 0 81612K 6844K semwai 2 0:00 2.05% 0.20% postgres 36368 pgsql 4 0 82416K 20780K sbwait 3 0:00 0.50% 0.15% postgres 36459 pgsql -4 0 81840K 7816K semwai 0 0:00 1.54% 0.15% postgres 36469 pgsql -4 0 81840K 7964K semwai 2 0:00 1.54% 0.15% postgres 36466 pgsql 129 0 81840K 7976K *Giant 2 0:00 1.54% 0.15% postgres 36479 pgsql -4 0 81528K 6648K semwai 0 0:00 3.00% 0.15% postgres 36457 pgsql -4 0 81840K 8040K semwai 1 0:00 1.03% 0.10% postgres 36450 pgsql 129 0 82352K 8188K *Giant 2 0:00 1.03% 0.10% postgres 36472 pgsql -4 0 81824K 7416K semwai 2 0:00 1.03% 0.10% postgres 36478 pgsql 131 0 81840K 7936K select 0 0:00 2.00% 0.10% postgres 36454 pgsql 4 0 82416K 16300K sbwait 3 0:00 0.51% 0.05% postgres 36414 pgsql 4 0 82416K 15872K sbwait 2 0:00 0.27% 0.05% postgres Our kernel is GENERIC plus: maxusers 512 options SYSVSHM options SHMMAXPGS=262144 options SHMSEG=512 options SHMMNI=512 options SYSVSEM options SEMMNI=512 options SEMMNS=1024 options SEMMNU=512 options SEMMAP=512 options NMBCLUSTERS=32768 Interesting bits from postgresql.conf: max_connections = 512 shared_buffers = 8192 sort_mem = 16384 vacuum_mem = 8192 fsync = false It seems that queries are executing fine once they start, but it's taking a while for them to get going, while the postgres process sits in semwait, sbwait or select. This problem doesn't happen when there's little load on the server, it's only when we open it for public consumption that it exhibits these problems. Anyone have this type of problem before? Am I missing something? Thanks, Jason From owner-freebsd-performance@FreeBSD.ORG Tue Mar 23 12:49:35 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 26FBE16A4CE for ; Tue, 23 Mar 2004 12:49:35 -0800 (PST) Received: from web14102.mail.yahoo.com (web14102.mail.yahoo.com [216.136.172.132]) by mx1.FreeBSD.org (Postfix) with SMTP id EEE4E43D55 for ; Tue, 23 Mar 2004 12:49:34 -0800 (PST) (envelope-from cguttesen@yahoo.dk) Message-ID: <20040323204927.35614.qmail@web14102.mail.yahoo.com> Received: from [194.248.174.33] by web14102.mail.yahoo.com via HTTP; Tue, 23 Mar 2004 21:49:27 CET Date: Tue, 23 Mar 2004 21:49:27 +0100 (CET) From: =?iso-8859-1?q?Claus=20Guttesen?= To: Jason Coene , freebsd-performance@freebsd.org In-Reply-To: <200403232002.i2NK2Gam077177@mail.gotfrag.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: Re: FreeBSD, PostgreSQL, semwait and sbwait! X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Mar 2004 20:49:35 -0000 > We're having a substantial problem with our FreeBSD > 5.2 database server > running PostgreSQL - it's getting a lot of traffic > (figure about 3,000 > queries per second), but queries are slow, and it's > seemingly waiting on > other things than CPU time. > ... > It seems that queries are executing fine once they > start, but it's taking a > while for them to get going, while the postgres It may not apply, but is DNS configured correct? Try connect with ssh and see if it takes a few sec's before you connect. Your postgres-info seems fine. As a sidenote you may want to copy GENERIC to a new file and compile that one instead. During your next cvsup GENERIC will be recovered to it's original state and your changes will be lost. regards Claus Yahoo! Mail (http://dk.mail.yahoo.com) - Gratis: 6 MB lagerplads, spamfilter og virusscan From owner-freebsd-performance@FreeBSD.ORG Tue Mar 23 13:12:01 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6EC2016A4D7 for ; Tue, 23 Mar 2004 13:12:01 -0800 (PST) Received: from rlx13.zapatec.com (66-117-144-213.zapatec.lmi.net [66.117.144.213]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2F31343D31 for ; Tue, 23 Mar 2004 13:12:01 -0800 (PST) (envelope-from dror@rlx11.zapatec.com) Received: from rlx11.zapatec.com (rlx11.pr.zapatec.com [192.168.1.132]) by rlx13.zapatec.com (Postfix) with ESMTP id EC422A941 for ; Tue, 23 Mar 2004 13:12:00 -0800 (PST) Received: (from dror@localhost) by rlx11.zapatec.com (8.12.3/8.12.3/Submit) id i2NLC0ih037363 for freebsd-performance@freebsd.org; Tue, 23 Mar 2004 13:12:00 -0800 (PST) (envelope-from dror) Date: Tue, 23 Mar 2004 13:12:00 -0800 From: Dror Matalon To: freebsd-performance@freebsd.org Message-ID: <20040323211200.GZ1757@rlx11.zapatec.com> References: <200403232002.i2NK2Gam077177@mail.gotfrag.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200403232002.i2NK2Gam077177@mail.gotfrag.com> Subject: Re: FreeBSD, PostgreSQL, semwait and sbwait! X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Mar 2004 21:12:01 -0000 Databases tend to be io bound not cpu bound. Try running something like iostat 5 and see how disk utilization varies between testing and production. If it's taking 3 seconds to run a query that should take 50ms, I would guess that it's fetching pages from disk. Also, I'd suggest asking this question with lots more information about the kind of queries you're running, on the postgresql-performance mailing list. There's a lot more traffic there and knowledgeable people. Dror On Tue, Mar 23, 2004 at 03:02:22PM -0500, Jason Coene wrote: > Hello all, > > We're having a substantial problem with our FreeBSD 5.2 database server > running PostgreSQL - it's getting a lot of traffic (figure about 3,000 > queries per second), but queries are slow, and it's seemingly waiting on > other things than CPU time. > > The database server is a dual P4-2.8 w/ HT enabled (kernel finds 4 > processors), 2GB RAM, 4 disk Serial ATA on 3ware RAID, gigabit Ethernet > connection to web servers. It's running FreeBSD 5.2 and PostgreSQL 7.4.1. > > The server is taking a while to respond to both connections, and then > queries (between 1-3 seconds, on a query that should execute in 0.05 or > less). > > The CPU usage for the server never goes above 30% (70% idle), and the CPU > time that's in use is nearly always split equal between user and system. > The system is using > > Doing a "top", this is what we see: > > last pid: 51833; load averages: 13.72, 11.74, 10.01 up 0+01:55:45 15:00:03 > 116 processes: 1 running, 99 sleeping, 16 lock > CPU states: 14.6% user, 0.0% nice, 23.7% system, 0.2% interrupt, 61.5% idle > Mem: 91M Active, 1043M Inact, 160M Wired, 52K Cache, 112M Buf, 644M Free > Swap: 4096M Total, 4096M Free > > 20354 pgsql 131 0 80728K 5352K select 0 0:24 1.71% 1.71% postgres > 36415 pgsql 4 0 81656K 67468K sbwait 2 0:00 3.23% 0.59% postgres > 36442 pgsql 128 0 82360K 15868K select 2 0:00 1.75% 0.24% postgres > 36447 pgsql -4 0 82544K 10616K semwai 0 0:00 2.05% 0.20% postgres > 36461 pgsql -4 0 81612K 6844K semwai 2 0:00 2.05% 0.20% postgres > 36368 pgsql 4 0 82416K 20780K sbwait 3 0:00 0.50% 0.15% postgres > 36459 pgsql -4 0 81840K 7816K semwai 0 0:00 1.54% 0.15% postgres > 36469 pgsql -4 0 81840K 7964K semwai 2 0:00 1.54% 0.15% postgres > 36466 pgsql 129 0 81840K 7976K *Giant 2 0:00 1.54% 0.15% postgres > 36479 pgsql -4 0 81528K 6648K semwai 0 0:00 3.00% 0.15% postgres > 36457 pgsql -4 0 81840K 8040K semwai 1 0:00 1.03% 0.10% postgres > 36450 pgsql 129 0 82352K 8188K *Giant 2 0:00 1.03% 0.10% postgres > 36472 pgsql -4 0 81824K 7416K semwai 2 0:00 1.03% 0.10% postgres > 36478 pgsql 131 0 81840K 7936K select 0 0:00 2.00% 0.10% postgres > 36454 pgsql 4 0 82416K 16300K sbwait 3 0:00 0.51% 0.05% postgres > 36414 pgsql 4 0 82416K 15872K sbwait 2 0:00 0.27% 0.05% postgres > > Our kernel is GENERIC plus: > > maxusers 512 > options SYSVSHM > options SHMMAXPGS=262144 > options SHMSEG=512 > options SHMMNI=512 > options SYSVSEM > options SEMMNI=512 > options SEMMNS=1024 > options SEMMNU=512 > options SEMMAP=512 > options NMBCLUSTERS=32768 > > Interesting bits from postgresql.conf: > > max_connections = 512 > shared_buffers = 8192 > sort_mem = 16384 > vacuum_mem = 8192 > fsync = false > > It seems that queries are executing fine once they start, but it's taking a > while for them to get going, while the postgres process sits in semwait, > sbwait or select. This problem doesn't happen when there's little load on > the server, it's only when we open it for public consumption that it > exhibits these problems. > > Anyone have this type of problem before? Am I missing something? > > Thanks, Jason > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" -- Dror Matalon Zapatec Inc 1700 MLK Way Berkeley, CA 94709 http://www.fastbuzz.com http://www.zapatec.com From owner-freebsd-performance@FreeBSD.ORG Tue Mar 23 13:48:36 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 610E216A4CE for ; Tue, 23 Mar 2004 13:48:36 -0800 (PST) Received: from mail2.dbitech.ca (radius.wavefire.com [64.141.13.252]) by mx1.FreeBSD.org (Postfix) with SMTP id 0C57F43D2F for ; Tue, 23 Mar 2004 13:48:36 -0800 (PST) (envelope-from darcy@wavefire.com) Received: (qmail 15065 invoked from network); 23 Mar 2004 21:55:28 -0000 Received: from dbitech.wavefire.com (HELO 64.141.15.253) (darcy@64.141.15.253) by radius.wavefire.com with SMTP; 23 Mar 2004 21:55:28 -0000 From: Darcy Buskermolen Organization: Wavefire Technologies Corp. To: "Jason Coene" , Date: Tue, 23 Mar 2004 13:48:34 -0800 User-Agent: KMail/1.5.4 References: <200403232002.i2NK2Gam077177@mail.gotfrag.com> In-Reply-To: <200403232002.i2NK2Gam077177@mail.gotfrag.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200403231348.34301.darcy@wavefire.com> Subject: Re: FreeBSD, PostgreSQL, semwait and sbwait! X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Mar 2004 21:48:36 -0000 On March 23, 2004 12:02 pm, Jason Coene wrote: > Hello all, > > We're having a substantial problem with our FreeBSD 5.2 database server > running PostgreSQL - it's getting a lot of traffic (figure about 3,000 > queries per second), but queries are slow, and it's seemingly waiting on > other things than CPU time. > > The database server is a dual P4-2.8 w/ HT enabled (kernel finds 4 > processors), 2GB RAM, 4 disk Serial ATA on 3ware RAID, gigabit Ethernet > connection to web servers. It's running FreeBSD 5.2 and PostgreSQL 7.4.1. > > The server is taking a while to respond to both connections, and then > queries (between 1-3 seconds, on a query that should execute in 0.05 or > less). What does the following query show (on the DB while it's under load)? SELECT r.relname,l.mode, count(*) AS numlocks FROM pg_locks l, pg_class r WHERE r.oid = l.relation GROUP BY relname,mode; if you have any sort of insert/updates happening, you may be waiting for a lock on that table to be available. Things like foreign keys make big use of locking while they CASCADE. > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org" -- Darcy Buskermolen Wavefire Technologies Corp. ph: 250.717.0200 fx: 250.763.1759 http://www.wavefire.com From owner-freebsd-performance@FreeBSD.ORG Tue Mar 23 13:52:20 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5CFE316A4CE for ; Tue, 23 Mar 2004 13:52:20 -0800 (PST) Received: from mxsf10.cluster1.charter.net (mxsf10.cluster1.charter.net [209.225.28.210]) by mx1.FreeBSD.org (Postfix) with ESMTP id EADA143D2F for ; Tue, 23 Mar 2004 13:52:19 -0800 (PST) (envelope-from paul@pathiakis.com) Received: from pathiaki.com (cpe-66-189-10-193.ma.charter.com [66.189.10.193]) i2NLkJX9083152 for ; Tue, 23 Mar 2004 16:46:19 -0500 (EST) (envelope-from paul@pathiakis.com) From: Paul Pathiakis To: freebsd-performance@freebsd.org Date: Tue, 23 Mar 2004 16:48:46 -0500 User-Agent: KMail/1.6.1 References: <200403232002.i2NK2Gam077177@mail.gotfrag.com> In-Reply-To: <200403232002.i2NK2Gam077177@mail.gotfrag.com> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200403231648.46566.paul@pathiakis.com> Subject: Re: FreeBSD, PostgreSQL, semwait and sbwait! X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Mar 2004 21:52:20 -0000 Hi, just a quick thought.... in /etc/sysctl.conf have you tried: kern.ipc.shm_use_phys=1 to tell shared memory to be mapped into physical memory space? P. On Tuesday 23 March 2004 03:02 pm, Jason Coene wrote: > Hello all, > > We're having a substantial problem with our FreeBSD 5.2 database server > running PostgreSQL - it's getting a lot of traffic (figure about 3,000 > queries per second), but queries are slow, and it's seemingly waiting on > other things than CPU time. > > The database server is a dual P4-2.8 w/ HT enabled (kernel finds 4 > processors), 2GB RAM, 4 disk Serial ATA on 3ware RAID, gigabit Ethernet > connection to web servers. It's running FreeBSD 5.2 and PostgreSQL 7.4.1. > > The server is taking a while to respond to both connections, and then > queries (between 1-3 seconds, on a query that should execute in 0.05 or > less). > > The CPU usage for the server never goes above 30% (70% idle), and the CPU > time that's in use is nearly always split equal between user and system. > The system is using > > Doing a "top", this is what we see: > > last pid: 51833; load averages: 13.72, 11.74, 10.01 up 0+01:55:45 15:00:03 > 116 processes: 1 running, 99 sleeping, 16 lock > CPU states: 14.6% user, 0.0% nice, 23.7% system, 0.2% interrupt, 61.5% idle > Mem: 91M Active, 1043M Inact, 160M Wired, 52K Cache, 112M Buf, 644M Free > Swap: 4096M Total, 4096M Free > > 20354 pgsql 131 0 80728K 5352K select 0 0:24 1.71% 1.71% > postgres 36415 pgsql 4 0 81656K 67468K sbwait 2 0:00 3.23% > 0.59% postgres 36442 pgsql 128 0 82360K 15868K select 2 0:00 1.75% > 0.24% postgres 36447 pgsql -4 0 82544K 10616K semwai 0 0:00 > 2.05% 0.20% postgres 36461 pgsql -4 0 81612K 6844K semwai 2 0:00 > 2.05% 0.20% postgres 36368 pgsql 4 0 82416K 20780K sbwait 3 > 0:00 0.50% 0.15% postgres 36459 pgsql -4 0 81840K 7816K semwai 0 > 0:00 1.54% 0.15% postgres 36469 pgsql -4 0 81840K 7964K semwai 2 > 0:00 1.54% 0.15% postgres 36466 pgsql 129 0 81840K 7976K *Giant > 2 0:00 1.54% 0.15% postgres 36479 pgsql -4 0 81528K 6648K > semwai 0 0:00 3.00% 0.15% postgres 36457 pgsql -4 0 81840K > 8040K semwai 1 0:00 1.03% 0.10% postgres 36450 pgsql 129 0 82352K > 8188K *Giant 2 0:00 1.03% 0.10% postgres 36472 pgsql -4 0 > 81824K 7416K semwai 2 0:00 1.03% 0.10% postgres 36478 pgsql 131 > 0 81840K 7936K select 0 0:00 2.00% 0.10% postgres 36454 pgsql 4 > 0 82416K 16300K sbwait 3 0:00 0.51% 0.05% postgres 36414 pgsql 4 > 0 82416K 15872K sbwait 2 0:00 0.27% 0.05% postgres > > Our kernel is GENERIC plus: > > maxusers 512 > options SYSVSHM > options SHMMAXPGS=262144 > options SHMSEG=512 > options SHMMNI=512 > options SYSVSEM > options SEMMNI=512 > options SEMMNS=1024 > options SEMMNU=512 > options SEMMAP=512 > options NMBCLUSTERS=32768 > > Interesting bits from postgresql.conf: > > max_connections = 512 > shared_buffers = 8192 > sort_mem = 16384 > vacuum_mem = 8192 > fsync = false > > It seems that queries are executing fine once they start, but it's taking a > while for them to get going, while the postgres process sits in semwait, > sbwait or select. This problem doesn't happen when there's little load on > the server, it's only when we open it for public consumption that it > exhibits these problems. > > Anyone have this type of problem before? Am I missing something? > > Thanks, Jason > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org" From owner-freebsd-performance@FreeBSD.ORG Tue Mar 23 16:51:13 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D0DF316A4CE for ; Tue, 23 Mar 2004 16:51:13 -0800 (PST) Received: from tornado.he.net (tornado.he.net [64.62.225.2]) by mx1.FreeBSD.org (Postfix) with SMTP id 9B41443D46 for ; Tue, 23 Mar 2004 16:51:13 -0800 (PST) (envelope-from wally@hotwally.com) Received: from [4.8.3.3] ([4.8.3.3]) by tornado.he.net for ; Tue, 23 Mar 2004 16:51:05 -0800 Mime-Version: 1.0 (Apple Message framework v612) Content-Transfer-Encoding: quoted-printable Message-Id: <54A9F806-7D2D-11D8-85F6-000A958F3FB6@hotwally.com> Content-Type: text/plain; charset=WINDOWS-1252; format=flowed To: freebsd-performance@freebsd.org From: wally morton Date: Tue, 23 Mar 2004 16:51:05 -0800 X-Mailer: Apple Mail (2.612) Subject: FTP fileserver??? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Mar 2004 00:51:14 -0000 I=92m looking for anyone who can give me some advice or their hard-won=20= knowledge on setting up FreeBSD to be a high-performance FTP (or http)=20= fileserver. I am involved in a project where I will probably need to=20 serve between 1,000 to 20,000 files per day of about 18Mb to a variety=20= of users connected at speeds ranging from DSL to (mostly) 28.8 K. Most=20= of these users will download 3 files totaling 18MB (each 6 MB). So it's=20= not a lot of data, but the same data is getting pumped out to many,=20 many users. So that=92s a daily traffic of 70-200GB transferred. Some of=20= the d/ls will be relatively fast over DSL, others slow. What is the=20 best way to set up FreeBSD to implement this? Just how much hardware do=20= I need? I don=92t want to overbuild, but I do want this server as fast=20= and solid as possible. I will connect it to a T1 or better. Any=20 suggestions welcomed. I was thinking this was going to be=20 dual-processor system with a lot of RAM and to keep as much of the=20 dataset in RAM as possible. Other ideas??Is dual CPU a waste of time?=20 Kernel and config changes??? I am not a very experienced FreeBSD=20 expert, but I'm no idiot either. Has no one written an article about=20 this? They must have. Please email responses to me directly as well as posting them here! I=20 am -- wally@hotwally.com= From owner-freebsd-performance@FreeBSD.ORG Tue Mar 23 17:31:37 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7F1D316A4CE for ; Tue, 23 Mar 2004 17:31:37 -0800 (PST) Received: from out012.verizon.net (out012pub.verizon.net [206.46.170.137]) by mx1.FreeBSD.org (Postfix) with ESMTP id 267EF43D2F for ; Tue, 23 Mar 2004 17:31:37 -0800 (PST) (envelope-from cswiger@mac.com) Received: from mac.com ([68.161.120.219]) by out012.verizon.net (InterMail vM.5.01.06.06 201-253-122-130-106-20030910) with ESMTP id <20040324013136.ZKWR18295.out012.verizon.net@mac.com>; Tue, 23 Mar 2004 19:31:36 -0600 Message-ID: <4060E4F3.6090103@mac.com> Date: Tue, 23 Mar 2004 20:31:31 -0500 From: Chuck Swiger Organization: The Courts of Chaos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: wally morton References: <54A9F806-7D2D-11D8-85F6-000A958F3FB6@hotwally.com> In-Reply-To: <54A9F806-7D2D-11D8-85F6-000A958F3FB6@hotwally.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Authentication-Info: Submitted using SMTP AUTH at out012.verizon.net from [68.161.120.219] at Tue, 23 Mar 2004 19:31:36 -0600 cc: freebsd-performance@freebsd.org Subject: Re: FTP fileserver??? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Mar 2004 01:31:37 -0000 wally morton wrote: > What is the best way to set up FreeBSD to implement this? Your requirements aren't demanding of anything special except lots of network bandwidth. Serving static content via FTP or HTTP is lightweight. > Just how much hardware do I need? A single-proc P3 system with RAID-10, -1, -5 disk config and a good power supply (redundant if you want to be fancy) should be plenty. > I will connect it to a T1 or better. You're probably going to need to; you mentioned "a daily traffic of 70-200GB", but a full T1 will only give you about 16GB/day (if my math is right). A 10/Mbs ethernet connection or a fractional T3, depending on what's available, is a better fit... > I was thinking this was going to be dual-processor system with a lot of > RAM and to keep as much of the dataset in RAM as possible. Other > ideas?? Is dual CPU a waste of time? Kernel and config changes??? Lots of RAM is a good idea to conserve disk I/O; dual-procs are a waste of money for such a low number of hits per day-- spend the bucks on getting good disks & RAID [preferably SCSI] instead. "man tuning" has some information but you may not need to do anything special to achieve reasonable performance for your application with a stock, untuned system. Have fun! :-) -- -Chuck From owner-freebsd-performance@FreeBSD.ORG Wed Mar 24 10:40:55 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DB3E416A4CF for ; Wed, 24 Mar 2004 10:40:55 -0800 (PST) Received: from tx0.oucs.ox.ac.uk (tx0.oucs.ox.ac.uk [129.67.1.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3A69443D31 for ; Wed, 24 Mar 2004 10:40:55 -0800 (PST) (envelope-from colin.percival@wadham.ox.ac.uk) Received: from scan0.oucs.ox.ac.uk ([129.67.1.162] helo=localhost) by tx0.oucs.ox.ac.uk with esmtp (Exim 4.24) id 1B6DJ8-0007v1-Di for performance@freebsd.org; Wed, 24 Mar 2004 18:40:54 +0000 Received: from rx0.oucs.ox.ac.uk ([129.67.1.161]) by localhost (scan0.oucs.ox.ac.uk [129.67.1.162]) (amavisd-new, port 25) with ESMTP id 30417-01 for ; Wed, 24 Mar 2004 18:40:54 +0000 (GMT) Received: from gateway.wadham.ox.ac.uk ([163.1.161.253]) by rx0.oucs.ox.ac.uk with smtp (Exim 4.24) id 1B6DJ8-0007uv-09 for performance@freebsd.org; Wed, 24 Mar 2004 18:40:54 +0000 Received: (qmail 18058 invoked by uid 1004); 24 Mar 2004 18:40:53 -0000 Received: from colin.percival@wadham.ox.ac.uk by gateway by uid 71 with qmail-scanner-1.20 (clamscan: 0.67. sweep: 2.18/3.79. Clear:RC:1(163.1.161.131):. Processed in 0.081856 secs); 24 Mar 2004 18:40:53 -0000 Received: from dhcp1131.wadham.ox.ac.uk (HELO piii600.wadham.ox.ac.uk) (163.1.161.131) by gateway.wadham.ox.ac.uk with SMTP; 24 Mar 2004 18:40:53 -0000 Message-Id: <6.0.1.1.1.20040324183111.043c2228@imap.sfu.ca> X-Sender: cperciva@imap.sfu.ca (Unverified) X-Mailer: QUALCOMM Windows Eudora Version 6.0.1.1 Date: Wed, 24 Mar 2004 18:40:51 +0000 To: Sean Chittenden From: Colin Percival In-Reply-To: References: <200403170937.i2H9bDvO033177@repoman.freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Mailman-Approved-At: Wed, 24 Mar 2004 12:06:43 -0800 cc: performance@freebsd.org cc: Colin Percival cc: Jason Coene Subject: Re: cvs commit: src/sys/kern sysv_sem.c X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Mar 2004 18:40:56 -0000 At 17:51 24/03/2004, Sean Chittenden wrote: >> FreeBSD src repository >> >> Modified files: >> sys/kern sysv_sem.c >> Log: >> Adjust the number of processes waiting on a semaphore properly if we're >> woken up in the middle of sleeping. > >Have you read the thread on performance@ regarding semaphore performance problems for PostgreSQL? Do you think this has any bearing on that problem? -sc I hadn't seen the thread on performance@ until now, but it sounds like this is very likely related -- this bug would result in processes not being woken up when a semaphore becomes free. Jason, could you update your src/sys/kern/sysv_sem.c to the latest revision (1.69), build a new kernel, and see if the problem goes away? Colin Percival