Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 28 Oct 2004 19:13:36 +0900
From:      Shunsuke SHINOMIYA <shino@fornext.org>
To:        John Baldwin <jhb@FreeBSD.org>
Cc:        obrien@FreeBSD.org
Subject:   Re[2]: disabling interrupt storm protection
Message-ID:  <20041028165604.DCF8.SHINO@fornext.org>
In-Reply-To: <200410271352.04069.jhb@FreeBSD.org>
References:  <20041022171924.73B3.SHINO@fornext.org> <200410271352.04069.jhb@FreeBSD.org>

next in thread | previous in thread | raw e-mail | index | archive | help

 Thank you, John.

> Can you try this patch:
> 
> --- //depot/vendor/freebsd/src/sys/kern/kern_intr.c	2004/09/05 02:10:52
> +++ //depot/projects/smpng/sys/kern/kern_intr.c	2004/10/27 17:41:06

 snip

> First, can you test that your system works with the default threshold, and 
> second, can you test that setting the threshold to zero disables the storm 
> detection?

 I applied your patch to kern_intr.c, and then I transmitted the high
 rate(148kpps) IP traffic to the box with the default threshold. But
 storm protection was not invoked.

 It seem that sensitivity of the detector is fallen.

 I tried some patterns of the packet loss test(transmitting the packets
 of the wire rate in 100Mbps to the box for 10 seconds.  The frame size
 of each packets is 64 octet).

 These results are as follows.
 
 Each `before' and `after' are outputs of `vmstat -i | grep em'. Please
 observe the first numerical value(number of interrupts since system
 startup). kern.hz is 100 at all trials. And `Interrupt Moderation', one
 of the feature of em was disabled.

==========
o kern_intr.c applied your patch.
o hw.intr_storm_threshold=500(default)

remark
 o The test results of both 1st and 2nd trial are good, low packet loss
 rate. A difference of both trials may be measurement error.
 o Interrupt storm protections is not invoked.
 o The interrupts rate of em0(input side) is about 25000 interrupts per
 second during a trial.
 o When hw.intr_storm_threshold=200, Interrupt storm protections is not
 invoked.

----------
1st trial
before
irq54: em0                             4          0
irq55: em1                            10          0

after
irq54: em0                        255247       2092
irq55: em1                        168688       1382

test result
Plr 0.875%

----------
2nd trial
before
irq54: em0                        255247       2092
irq55: em1                        168688       1382

after
irq54: em0                        510332       2883
irq55: em1                        337599       1907

result
Plr 0.957%

==========
o kern_intr.c applied your patch and compiled with -DHACK flags.
 (it cancel the your hack. is it right?)
o hw.intr_storm_threshold=500(default)

remark
 o The test results of 1st to 3rd trials are bad, high packet loss rate.
 During these tirals, em0 may be in the warming up phase and interrupt
 storm detection was invoked(the console output `Interrupt storm
 detected'). Because the number of interrupts since system startup is
 less than initial `warming' value(= threshold * 10).
 o I want to avoid this phenomenon.
 o The result of 4th trial is higher plr than 5th trial. My guess is
 that em0 was in the warming phase at the forepart of 4th trial.

----------
1st trial
before
irq54: em0                             3          0
irq55: em1                             8          0
after
irq54: em0                          1577          4
irq55: em1                          9302         25

result
Plr 84.037%

----------
2nd trial
before
irq54: em0                          1577          4
irq55: em1                          9302         25

after
irq54: em0                          3150          6
irq55: em1                         19584         39

result
Plr 84.428%

----------
3rd trial
before
irq54: em0                          3150          6
irq55: em1                         19584         39

after
irq54: em0                          4721          8
irq55: em1                         30127         53

result
Plr 83.855%

----------
4th trial
before
irq54: em0                          4721          8
irq55: em1                         30127         53

after
irq54: em0                        269893        420
irq55: em1                        196292        305

result
Plr 0.492%

----------
5th trial
before
irq54: em0                        269893        420
irq55: em1                        196292        305

after
irq54: em0                        532196        719
irq55: em1                        358797        484

result
Plr 0.361%

==========
o kern_intr.c applied your patch and compiled with -DHACK flags.
o hw.intr_storm_threshold=0

remark
 o The test result of 1st trial is worse than 2nd one. I think the cause is
 em0 in warming up phase at the forpart of the trial.

----------
1st trial
before
irq54: em0                             3          0
irq55: em1                             8          0
after
irq54: em0                        275850       1319
irq55: em1                        180104        861

result
Plr 1.133%

----------
2nd trial
before
irq54: em0                        275850       1319
irq55: em1                        180104        861

after
irq54: em0                        548886       1932
irq55: em1                        355397       1251

result
Plr 0.937%

----------
3rd trial
before
irq54: em0                        548886       1932
irq55: em1                        355397       1251

after
irq54: em0                        821117       2280
irq55: em1                        531581       1476

result
Plr 0.902%


-- 
Shunsuke SHINOMIYA <shino@fornext.org>



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20041028165604.DCF8.SHINO>