Date: Wed, 17 Oct 2007 15:41:22 -0400 From: kurtseel <kurtseel@primetime.com> To: Patrick Tracanelli <eksffa@freebsdbrasil.com.br> Cc: freebsd-geom@freebsd.org Subject: Re: gmirror + ggated question Message-ID: <47166562.60803@primetime.com> In-Reply-To: <47165C0B.7080707@freebsdbrasil.com.br> References: <471650AA.30903@primetime.com> <47165C0B.7080707@freebsdbrasil.com.br>
next in thread | previous in thread | raw e-mail | index | archive | help
Patrick Tracanelli wrote: > kurtseel escreveu: >> >> I built a mirror of a local drive and a ggated backed device. I ran >> iozone on it >> and it runs along fine until a certain point when it slows down to a >> near stand >> still. It doesn't break the mirror or crash the system, but it does >> slow the system >> down to a near stop. >> I kill the iozone, and a short time later I can login and then : >> >> # df >> Filesystem 1K-blocks Used Avail Capacity Mounted on >> /dev/mirror/thinkcs1a 1012974 155780 776158 17% / >> devfs 1 1 0 100% /dev >> /dev/mirror/thinkcs1e 85469448 1163474 77468420 1% /usr >> /dev/mirror/thinkcs1d 4058062 40426 3692992 1% /var >> [root@ ~/temp]# gmirror status >> Name Status Components >> mirror/thinkc COMPLETE ad0 >> ggate0 >> >> And all seems normal again. Seems like it has to do with big files ... >> This is the same configuration I used in : >> http://bsdtips.utcorp.net/mediawiki/index.php/Mirroring_over_network >> This is where the iozone gets stuck : > > Did you try raising send and receive buffers on ggated? I found myself > confortable with -S and -R around 512k-780k. I didnt, however, did an > iozone stress test, just a production test (real load) before going > production. > > Try raising the buffer and let us know about your tests. TCP_NODELAY > is also worth trying. > Makes sense. So now I get this : Test (/root/benchmarks) > ggated -v -R 262144 -S 262144 /etc/ggated.conf info: Reading exports file (/etc/ggated.conf). debug: Added 10.200.1.200/32 /dev/ad10 RW to exports list. info: Exporting 1 object(s). error: Cannot open stream socket: No buffer space available. error: Exiting. Test (/root/benchmarks) > ggated -v -R 524288 -S 524288 /etc/ggated.conf info: Reading exports file (/etc/ggated.conf). debug: Added 10.200.1.200/32 /dev/ad10 RW to exports list. info: Exporting 1 object(s). error: Cannot open stream socket: No buffer space available. error: Exiting. I have raised sysctl net.inet.tcp.sendspace=4194304 sysctl net.inet.tcp.recvspace=4194304 sysctl kern.ipc.maxsockbuf=2097152 Which I saw in a posting ... It even happens here : Test (/root/benchmarks) > ggated -v -R 1 -S 1 /etc/ggated.conf info: Reading exports file (/etc/ggated.conf). debug: Added 10.200.1.200/32 /dev/ad10 RW to exports list. info: Exporting 1 object(s). error: Cannot open stream socket: No buffer space available. error: Exiting.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?47166562.60803>