From owner-freebsd-fs@FreeBSD.ORG Fri Jun 30 03:57:07 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D30E116A412 for ; Fri, 30 Jun 2006 03:57:07 +0000 (UTC) (envelope-from leo.huang.gd@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.170]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2106A43D49 for ; Fri, 30 Jun 2006 03:57:06 +0000 (GMT) (envelope-from leo.huang.gd@gmail.com) Received: by ug-out-1314.google.com with SMTP id m3so719798uge for ; Thu, 29 Jun 2006 20:57:05 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=X2I9SW3ndHF+Ab48OjBABxmvkyPXEl10W5ltk1RCQzqf6HL+LMJL6MCgvgUjqhbzyJCYx5eEXWaIe+nuX0zByp6zlTu1SEO2g9Vt9W/3VxR371VY0lHhPprcMLrCL/MAz8jXomSmG5KiBWKQTJ9Dp81t5BlMABM32oy773/TLhE= Received: by 10.67.101.8 with SMTP id d8mr2630211ugm; Thu, 29 Jun 2006 20:57:05 -0700 (PDT) Received: by 10.67.27.12 with HTTP; Thu, 29 Jun 2006 20:57:05 -0700 (PDT) Message-ID: Date: Fri, 30 Jun 2006 11:57:05 +0800 From: "Leo Huang" To: "Bruce Evans" In-Reply-To: <20060629201157.N77878@delplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <44A1B958.4030204@fer.hr> <20060628230439.M75051@delplex.bde.org> <20060629201157.N77878@delplex.bde.org> Cc: freebsd-fs@freebsd.org, Ivan Voras Subject: Re: Is the fsync() fake on FreeBSD6.1? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Jun 2006 03:57:07 -0000 hi, Bruce > How did you disable "driver" write caching? I think both Bjorn and I > meant the _drive_ write caching, and that is what you refer to as "disk" > write caching. Only turn off the caching in the lowest layer (disk == > drive). Yes, "disk" what I say means drive. On FreeBSD, I use camcontrol to disable the drive write caching. Enable: mysql-test-4# camcontrol modepage da0 -m8 IC: 0 ABPF: 0 CAP: 0 DISC: 1 SIZE: 0 WCE: 1 MF: 0 RCD: 0 Demand Retention Priority: 0 Write Retention Priority: 0 Disable Pre-fetch Transfer Length: 65535 Minimum Pre-fetch: 0 Maximum Pre-fetch: 0 Maximum Pre-fetch Ceiling: 65535 Disable: mysql-test-4# camcontrol modepage da0 -m8 IC: 0 ABPF: 0 CAP: 0 DISC: 1 SIZE: 0 WCE: 0 MF: 0 RCD: 0 Demand Retention Priority: 0 Write Retention Priority: 0 Disable Pre-fetch Transfer Length: 65535 Minimum Pre-fetch: 0 Maximum Pre-fetch: 0 Maximum Pre-fetch Ceiling: 65535 On Debian, I use sginfo to disable it. Enable: mysql-test-1:/home/huangjy# sginfo -c /dev/sda Caching mode page (0x8) ----------------------- Initiator Control 0 ABPF 0 CAP 0 DISC 1 SIZE 0 Write Cache Enabled 1 MF 0 Read Cache Disabled 0 Demand Read Retention Priority 0 Demand Write Retention Priority 0 Disable Pre-fetch Transfer Length 65535 Minimum Pre-fetch 0 Maximum Pre-fetch 0 Maximum Pre-fetch Ceiling 65535 FSW 1 LBCSS 0 DRA 0 Number of Cache Segments 8 Cache Segment size 0 Non-Cache Segment size 0 Disable: mysql-test-1:/home/huangjy# sginfo -c /dev/sda Caching mode page (0x8) ----------------------- Initiator Control 0 ABPF 0 CAP 0 DISC 1 SIZE 0 Write Cache Enabled 0 MF 0 Read Cache Disabled 0 Demand Read Retention Priority 0 Demand Write Retention Priority 0 Disable Pre-fetch Transfer Length 65535 Minimum Pre-fetch 0 Maximum Pre-fetch 0 Maximum Pre-fetch Ceiling 65535 FSW 1 LBCSS 0 DRA 0 Number of Cache Segments 8 Cache Segment size 0 Non-Cache Segment size 0 > I wonder when all drives will have enough fast enough nonvolatile RAM for > write caching to just work. Our test computer only have one scsi disk. So I think that the data in drive write caching will be lost when the power is off. Regards, Leo Huang 2006/6/29, Bruce Evans : > On Thu, 29 Jun 2006, Leo Huang wrote: > > >> >> OS Clients Result(queries per second) TPS(got from > >> >> iostat) > >> >> FreeBSD6.1 50 516.1 about 2000 > >> > >> Seems normal for drives that do write caching. > > > > I disable the driver write caching as Bjorn Gronvall suggest, the > > result show that the TPS come down to about 200. So I think you and > > Bjorn Gronvall are right. It is the disk write caching make the TPS so > > high. > > > >> >> Debian3.1 50 49.8 about 200 > >> > >> Seems to slow for disks that do write caching. Maybe Debian does something > >> to force the drive to complete it's i/o, or just does a full sync() like > >> someone mentioned Linux doing. > > > > I use sginfo the find that the disk write caching is also enabled > > default. After the disk write caching is disabled, the TPS also come > > down from 200 to 110. This is really pullze me. Can you give me more > > infomation about it? > > How did you disable "driver" write caching? I think both Bjorn and I > meant the _drive_ write caching, and that is what you refer to as "disk" > write caching. Only turn off the caching in the lowest layer (disk == > drive). > > I wonder when all drives will have enough fast enough nonvolatile RAM for > write caching to just work. > > Bruce >