From owner-freebsd-stable@FreeBSD.ORG  Mon Jan 21 11:33:25 2013
Return-Path: <owner-freebsd-stable@FreeBSD.ORG>
Delivered-To: freebsd-stable@freebsd.org
Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115])
 by hub.freebsd.org (Postfix) with ESMTP id 62ADA134
 for <freebsd-stable@freebsd.org>; Mon, 21 Jan 2013 11:33:25 +0000 (UTC)
 (envelope-from danny@cs.huji.ac.il)
Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.16.84])
 by mx1.freebsd.org (Postfix) with ESMTP id 1CE70A2A
 for <freebsd-stable@freebsd.org>; Mon, 21 Jan 2013 11:33:24 +0000 (UTC)
Received: from pampa.cs.huji.ac.il ([132.65.80.32])
 by kabab.cs.huji.ac.il with esmtp
 id 1TxFcr-0006dx-MX; Mon, 21 Jan 2013 13:33:17 +0200
X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.3
To: freebsd-stable@freebsd.org
Subject: time issues and ZFS
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Date: Mon, 21 Jan 2013 13:33:17 +0200
From: Daniel Braniss <danny@cs.huji.ac.il>
Message-ID: <E1TxFcr-0006dx-MX@kabab.cs.huji.ac.il>
Cc: Ronald Klop <ronald-freebsd8@klop.yi.org>
X-BeenThere: freebsd-stable@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Production branch of FreeBSD source code <freebsd-stable.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-stable>,
 <mailto:freebsd-stable-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-stable>
List-Post: <mailto:freebsd-stable@freebsd.org>
List-Help: <mailto:freebsd-stable-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-stable>,
 <mailto:freebsd-stable-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 21 Jan 2013 11:33:25 -0000

After many trials (and errors), here are some facts:

host: DELL PowerEdge R710, 16GB, 
 mfi0: <Dell PERC H700 Integrated>
 mfid0: 14305280MB (29297213440 sectors) RAID volume 'r5' is optimal
 mfi1: <Dell PERC 6> 
 mfid1: 12393472MB (25381830656 sectors) RAID volume 'Virtual Disk 0' is 
optimal

we have NO problems with FreeBSD-8.3-STABLE, but with 9.1-STABLE, the real-time
clock slows down when doing some zfs stuff like send|receive, typing 'date' 
when less that 1000s went by seems to crorrect the problem,
ntpd kicks in and on track again.

I have a cron job just logging date every 5 minutes, and the loghost sees:

|-- local time on loghost             | time on problematic host
Jan 20 19:56:19 store-02.cs.huji.ac.il Jan 20 19:56:19 danny: Sun Jan 20 
19:56:19 IST 2013	-- ok
Jan 20 20:15:00 store-02.cs.huji.ac.il Jan 20 20:15:00 danny: Sun Jan 20 
20:15:00 IST 2013	-- ok
Jan 20 21:30:00 store-02.cs.huji.ac.il Jan 20 20:21:06 danny: Sun Jan 20 
20:21:06 IST 2013	-- off by 1:09
Jan 20 21:33:53 store-02.cs.huji.ac.il Jan 20 20:25:00 danny: Sun Jan 20 
20:25:00 IST 2013	-- off by 1:08
Jan 20 21:38:54 store-02.cs.huji.ac.il Jan 20 20:30:00 danny: Sun Jan 20 
20:30:00 IST 2013	-- off by 1:09
...
Jan 20 22:03:54 store-02.cs.huji.ac.il Jan 20 20:55:00 danny: Sun Jan 20 
20:55:00 IST 2013	-- diff is now constant
..
Jan 20 22:04:13 store-02.cs.huji.ac.il Jan 20 20:55:19 ntpd[1848]: time 
correction of 4134 seconds exceeds sanity limit (1000); set clock manually to 
the correct UTC time.
...
Jan 20 22:58:53 store-02.cs.huji.ac.il Jan 20 21:50:00 danny: Sun Jan 20 
21:50:00 IST 2013


strangely, when running 8.3, ACPI-fast is chosen:
	kern.timecounter.choice: TSC(-100) HPET(900) ACPI-fast(1000) i8254(0) 
dummy(-1000000)
but with 9.1 TSC-low gets chosen:
	kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) HPET(950) i8254(0) 
dummy(-1000000)

so I did sysctl kern.timecounter.hardware=ACPI-fast, but the same happens - 
unless it can't be changed after boot.

I realy need help here!

thanks,
	danny