From owner-freebsd-current@FreeBSD.ORG Fri Nov 18 21:49:19 2011 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 305A4106566C; Fri, 18 Nov 2011 21:49:19 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id DFB848FC14; Fri, 18 Nov 2011 21:49:18 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 1DE5D7300A; Fri, 18 Nov 2011 23:04:58 +0100 (CET) Date: Fri, 18 Nov 2011 23:04:58 +0100 From: Luigi Rizzo To: Doug Barton Message-ID: <20111118220458.GA21152@onelab2.iet.unipi.it> References: <201111180800.06593.jhb@freebsd.org> <20111118170615.GA9762@onelab2.iet.unipi.it> <201111181220.04846.jhb@freebsd.org> <20111118175433.GA13459@onelab2.iet.unipi.it> <4EC6AEF0.1010402@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4EC6AEF0.1010402@FreeBSD.org> User-Agent: Mutt/1.4.2.3i Cc: Matteo Landi , freebsd-current@freebsd.org Subject: Re: ixgbe and fast interrupts X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Nov 2011 21:49:19 -0000 On Fri, Nov 18, 2011 at 11:16:00AM -0800, Doug Barton wrote: > On 11/18/2011 09:54, Luigi Rizzo wrote: > > One more thing (i am mentioning it here for archival purposes, > > as i keep forgetting to test it). Is entropy harvesting expensive ? > > No. It was designed to be inexpensive on purpose. :) hmmm.... unfortunately I don't have a chance to test it until monday (probably one could see if the ping times change by modifying the value of kern.random.sys.harvest.* ). But in the code i see the following: - the harvest routine is this: void random_harvest(void *entropy, u_int count, u_int bits, u_int frac, enum esource origin) { if (reap_func) (*reap_func)(get_cyclecount(), entropy, count, bits, frac, origin); } - the reap_func seems to be bound to dev/random/randomdev_soft.c::random_harvest_internal() which internally uses a spinlock and then moves entries between two lists. I am concerned that the get_cyclecount() might end up querying an expensive device (is it using kern.timecounter.hardware ?) > sysctl -a | grep timecounter kern.timecounter.tick: 1 kern.timecounter.choice: TSC(-100) HPET(900) ACPI-fast(1000) i8254(0) dummy(-1000000) kern.timecounter.hardware: ACPI-fast So between the indirect function call, spinlock, list manipulation and the cyclecounter i wouldn't be surprised it the whole thing takes a microsecond or so. Anyways, on monday i'll know better. in the meantime, if someone wants to give it a try... in our tests between two machines and ixgbe (10G) interfaces, an unmodified 9.0 kernel has a median ping time of 30us with "slow" pings (say -i 0.01 or larger) and 17us with a ping -f . BTW the reason for the difference is totally unclear to me (ping -f uses a non-blocking select() but i don't think it can explain such a large delta). cheers luigi