Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 18 Nov 2011 23:04:58 +0100
From:      Luigi Rizzo <rizzo@iet.unipi.it>
To:        Doug Barton <dougb@FreeBSD.org>
Cc:        Matteo Landi <matteo@matteolandi.net>, freebsd-current@freebsd.org
Subject:   Re: ixgbe and fast interrupts
Message-ID:  <20111118220458.GA21152@onelab2.iet.unipi.it>
In-Reply-To: <4EC6AEF0.1010402@FreeBSD.org>
References:  <CALJ8J_HPZewO12uanb=kctQYwepMssr63E0DQh9CqV6PGaC=JA@mail.gmail.com> <201111180800.06593.jhb@freebsd.org> <20111118170615.GA9762@onelab2.iet.unipi.it> <201111181220.04846.jhb@freebsd.org> <20111118175433.GA13459@onelab2.iet.unipi.it> <4EC6AEF0.1010402@FreeBSD.org>

next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Nov 18, 2011 at 11:16:00AM -0800, Doug Barton wrote:
> On 11/18/2011 09:54, Luigi Rizzo wrote:
> > One more thing (i am mentioning it here for archival purposes,
> > as i keep forgetting to test it). Is entropy harvesting expensive ?
> 
> No. It was designed to be inexpensive on purpose. :)

hmmm....
unfortunately I don't have a chance to test it until monday
(probably one could see if the ping times change by modifying
the value of kern.random.sys.harvest.* ).

But in the code i see the following:

- the harvest routine is this:

    void
    random_harvest(void *entropy, u_int count, u_int bits, u_int frac,
	enum esource origin)
    {
        if (reap_func)
                (*reap_func)(get_cyclecount(), entropy, count, bits, frac,
                    origin);
    }

- the reap_func seems to be bound to

    dev/random/randomdev_soft.c::random_harvest_internal()

  which internally uses a spinlock and then moves entries between
  two lists.

I am concerned that the get_cyclecount() might end up querying an
expensive device (is it using kern.timecounter.hardware ?)

    > sysctl -a | grep timecounter
    kern.timecounter.tick: 1
    kern.timecounter.choice: TSC(-100) HPET(900) ACPI-fast(1000) i8254(0) dummy(-1000000)
    kern.timecounter.hardware: ACPI-fast

So between the indirect function call, spinlock, list manipulation
and the cyclecounter i wouldn't be surprised it the whole thing
takes a microsecond or so.

Anyways, on monday i'll know better. in the meantime, if someone
wants to give it a try... in our tests between two machines and
ixgbe (10G) interfaces, an unmodified 9.0 kernel has a median ping
time of 30us with "slow" pings (say -i 0.01 or larger) and 17us with
a ping -f .
BTW the reason for the difference is totally unclear to me (ping
-f uses a non-blocking select() but i don't think it can explain
such a large delta).

	cheers
	luigi



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20111118220458.GA21152>