Skip site navigation (1)Skip section navigation (2) 
Date:      Mon, 25 Mar 2019 12:16:52 -0500
From:      Karl Denninger <karl@denninger.net>
To:        Ian Lepore <ian@freebsd.org>, ticso@cicely.de
Cc:        "freebsd-arm@freebsd.org" <freebsd-arm@freebsd.org>
Subject:   Re: Options for FBSD support with LCD device - new project [[Maybe related: I2c issues on the Pi2]]
Message-ID:  <d96c7f42-f01b-8990-a558-ee92d631b51d@denninger.net>
In-Reply-To: <fc17ac0f77832e840b9fffa9b1074561f1e766d8.camel@freebsd.org>
References:  <004ddba628b94b80845d8e509ddcb648d21fd6c9.camel@freebsd.org> <C68D7E6E-03C1-448F-8638-8BD1717DBF44@jeditekunum.com> <ac7d434f16f3a89f5ef247678d6becdbeded5c3f.camel@freebsd.org> <CE40E2B5-2244-4EF9-B67F-34A54D71E2E8@jeditekunum.com> <f60ea6d2-b696-d896-7bcb-ac628f41f7b8@denninger.net> <20190319161423.GH57400@cicely7.cicely.de> <52df098fdc0caf5de1879c93239534fffbd49b56.camel@freebsd.org> <40f57de2-2b25-3981-a416-b9958cc97636@denninger.net> <669892ac3fc37b0843a156c0ab102316829103fd.camel@freebsd.org> <663f2566-b035-7011-70eb-4163b41e6e55@denninger.net> <20190325164827.GL57400@cicely7.cicely.de> <3db9cf8a-68ee-e339-67bf-760ee51464fd@denninger.net> <fc17ac0f77832e840b9fffa9b1074561f1e766d8.camel@freebsd.org>
index | next in thread | previous in thread | raw e-mail 

[-- Attachment #1 --]
On 3/25/2019 12:05, Ian Lepore wrote:
> On Mon, 2019-03-25 at 11:58 -0500, Karl Denninger wrote:
>> On 3/25/2019 11:48, Bernd Walter wrote:
>>> On Mon, Mar 25, 2019 at 11:33:32AM -0500, Karl Denninger wrote:
>>>>> What do you mean by an insane rate?  It's normal for the usb
>>>>> controller
>>>>> to be showing around thousands of int/sec.  Despite what seems
>>>>> like a
>>>>> high rate, even on an on rpi-b it uses under 2% cpu to service
>>>>> that.
>>>>>
>>>>> root@rpi:~ # vmstat -i
>>>>> interrupt                        total       rate
>>>>> intc0,2: vchiq0                      2          0
>>>>> intc0,11: systimer0           10103206       1110
>>>>> intc0,17:-x_dwcotg0          218596055      24007
>>>>> intc0,28: bcm_dma0                 834          0
>>>>> intc0,61: iichb0                  5778          1
>>>>> intc0,65: uart0                   1817          0
>>>>> intc0,70:-dhci_bcm0                172          0
>>>>> Total                        228707864      25118
>>>>>
>>>>> -- Ian
>>>> The story gets more odd.
>>>>
>>>> The same *physical* unit that I saw this on last night with no
>>>> I2c
>>>> device connected I restarted this morning -- changing NOTHING --
>>>> and it
>>>> disappeared.
>>>>
>>>> But -- on another unit it's still there (I haven't shut down,
>>>> pulled
>>>> power and restarted that one.)
>>>>
>>>> vmstat -i on both doesn't show anything all that odd:
>>>> misbehaving that's not there, and neither are the missed
>>>> interrupt
>>>> complaints.
>>>>
>>>> But again, last night the one that this morning is NOT
>>>> misbehaving WAS,
>>>> and was showing the exact same thing.
>>>>
>>>> So this looks like something that is not being initialized
>>>> property at
>>>> boot time, and sometimes however it comes up causes trouble, and
>>>> other
>>>> times it does not -- which is likely to make it a "lot" of fun to
>>>> find.
>>> By causing trouble - do you mean it doesn't work?
>>> I noticed that my system has this message:
>>> nxprtc0: RTC clock not running
>>> Warning: bad time from time-of-day clock, system time will not be
>>> set accurately
>>> This shouldn't happen, but I wonder if the iic communication works
>>> at all.
>>> I likely wouldn't notice if the rtc failed.
>>> Maybe there was an initial problem at start as you said.
>>> Will reboot it and see what happens.
>>> After a reboot the message about the rtc is gone.
>>> Have to wait at least a day to see if the Spurious are gone too.
>> In both cases on my boxes everything is working, but that's not
>> unexpected because of the way my code works (it dynamically detects a
>> change in configuration in that if it tries to open the I2c bus when
>> there's a configuration file for devices on it, and it fails, it will
>> try again in a few seconds -- and if you remove the config then it
>> will
>> shut down the I/O path in a short while and stop.)
>>
>> On the units that exhibit the problem the load average is 1.0 +
>> whatever
>> is real *and* the crazy interrupt rate is present.  On the ones that
>> are
>> not neither is the case; the native and real load average is present
>> and
>> the interrupt rate is normal.
>>
>> In the case of the unit that the problem showed up on and then
>> disappeared, however, while there's an I2c config defined there's no
>> device connected to it on my bench.
>>
>> But I suspect this is something banging the interrupts on the CPU
>> that
>> is not attached to anything in the code, and the reason I suspect
>> that
>> is that on a given boot it either happens or not, and if it does then
>> nothing I can do will make it stop -- and likewise, nothing I can do
>> will make it start if it doesn't on boot.  That implies that whatever
>> it
>> is it's not code-specific nor .ko-loaded specific either, in that if
>> it
>> was related specifically to an I2c device being talked to actively
>> then
>> when I killed the code that was using I2c or booted without the
>> device
>> connected (or never started the code that attempted to probe the bus
>> and
>> attach to the device in question) it wouldn't do it at all -- but
>> that's
>> not true.
>>
>> The one that stopped doing it I then attached both an I2c device that
>> it
>> was looking for and also connected a "modem-style" device (which
>> caused
>> umodem.ko to autoload, as expected) and it came up as well, without a
>> problem -- and without triggering the mad interrupt storm.
>>
> Is the interrupt rate consistent from second to second?  Running
> 'vmstat 1' for a while might be useful to see that.  That many
> interrupts almost sounds like a line is floating, but if that were the
> case I'd expect a widely varying number of int/sec.
>
> If you build custom kernels, it might be helpful to apply r345475
> locally... it will display partial device names instead of just '+'
> when the name doesn't fit in the vmstat output.
>
> -- Ian

No, but it's in the same general range -- around 500k although it does
flop around some, and occasionally by a lot (e.g. if I sit and watch it
it'll occasionally put up VERY different numbers -- e.g. a ~730k number,
then it goes back, etc.)

I don't generally build custom kernels on these but I CAN put this into
the STABLE tree I'm building that from since I keep a separate one for
Crochet builds on these boxes.  Where do I find that specific delta?  (I
usually just svn things and I don't want to roll it all the way back to
there, right -- or do I?)

-- 
Karl Denninger
karl@denninger.net <mailto:karl@denninger.net>
/The Market Ticker/
/[S/MIME encrypted email preferred]/

[-- Attachment #2 --]
0	*H
010
	`He0	*H

00H^Ōc!5
H0
	*H
010	UUS10UFlorida10U	Niceville10U
Cuda Systems LLC10UCuda Systems CA1!0UCuda Systems LLC 2017 CA0
170817164217Z
270815164217Z0{10	UUS10UFlorida10U
Cuda Systems LLC10UCuda Systems CA1%0#UCuda Systems LLC 2017 Int CA0"0
	*H
0
h-5B>[;olӴ0~͎O9}9Ye*$g!ukvʶLzN`jL>MD'7U45CB+kY`bd~b*c3Ny-78ju]9HeuέsӬDؽmgwER?&UURj'}9nWD i`XcbGz\gG=u%\Oi13ߝ4
K44pYQr]Ie/r0+eEޝݖ0C15Mݚ@JSZ(zȏNTa(25DD5.l<g[[ZarQQ%Buȴ~~`IohRbʳڟu2MS8EdFUClCMaѳ!}ș+2k/bųE,n当ꖛ\(8WV8	d]b	yXw	܊:I39
00U]^§Q\ӎ0U#0T039N0b010	UUS10UFlorida10U	Niceville10U
Cuda Systems LLC10UCuda Systems CA1!0UCuda Systems LLC 2017 CA	@Ui0U00U0
	*H
:P U!>vJnio-#ן]WyujǑR̀Q
nƇ!GѦFg\yLxgw=OPycehf[}ܷ['4ڝ\[p6\o.B&JF"ZC{;*o*mcCcLY߾`
t*S!񫶭(`]DHP5A~/NPp6=mhk밣'doA$86hm5ӚS@jެEgl
)0JG`%k35PaC?σ
׳HEt}!P㏏%*BxbQwaKG$6h¦Mve;[o-Iی&
I,Tcߎ#t wPA@l0P+KXBպT	zGv;NcI3&JĬUPNa?/%W6G۟N000k#Xd\=0
	*H
0{10	UUS10UFlorida10U
Cuda Systems LLC10UCuda Systems CA1%0#UCuda Systems LLC 2017 Int CA0
170817212120Z
220816212120Z0W10	UUS10UFlorida10U
Cuda Systems LLC10Ukarl@denninger.net0"0
	*H
0
T[I-ΆϏdn;Å@שy.us~_ZG%<MYd\gvfnsa1'6Egyjs"C [{~_KPn+<*pv#Q+H/7[-vqDV^U>f%GX)H.|l`M(Cr>е͇6#odc"YljҦln8@5SA0&ۖ"OGj?UDWZ5	dDB7k-)9Izs-JAv
J6L$Ն1SmY.Lqw*SH;EF'DĦH]MOgQQ|Mٙג2Z9y@y]}6ٽeY9Y2xˆ$T=eCǺǵbn֛{j|@LLt1[Dk5:$=	`	M00<+00.0,+0 http://ocsp.cudasystems.net:88880	U00	`HB0U0U%0++03	`HB
&$OpenSSL Generated Client Certificate0U%՞V=؁;bzQ0U#0]^§Q\ӎϡ010	UUS10UFlorida10U	Niceville10U
Cuda Systems LLC10UCuda Systems CA1!0UCuda Systems LLC 2017 CAH^Ōc!5
H0U0karl@denninger.net0
	*H
۠A0-j%--$%g2#ޡ1^>{K+uGEv1ş7Af&b&O;.;A5*U)ND2bF|\=]<sˋL!wrw٧>YMÄ3\mWR hSv!_zvl? 3_ xU%\^#O*Gk̍YI_&Fꊛ@&1n”} ͬ:{hTP3B.;bU8:Z=^Gw8!k-@xE@i,+'Iᐚ:fhztX7/(hY` O.1}a`%RW^akǂpCAufgDixUTЩ/7}%=jnVZvcF<M=
2^GKH5魉
_O4ެByʈySkw=5@h.0z>
W1000{10	UUS10UFlorida10U
Cuda Systems LLC10UCuda Systems CA1%0#UCuda Systems LLC 2017 Int CAk#Xd\=0
	`HeE0	*H
	1	*H
0	*H
	1
190325171652Z0O	*H
	1B@q*e8*0Ub٭"jXa
<49+}<GQ 100l	*H
	1_0]0	`He*0	`He0
*H
0*H
0
*H
@0+0
*H
(0	+7100{10	UUS10UFlorida10U
Cuda Systems LLC10UCuda Systems CA1%0#UCuda Systems LLC 2017 Int CAk#Xd\=0*H
	10{10	UUS10UFlorida10U
Cuda Systems LLC10UCuda Systems CA1%0#UCuda Systems LLC 2017 Int CAk#Xd\=0
	*H
zk	]̶Y7du}Bra/3P} yv=H^l@`o1yj͗7D}Aq=(ט-4{z=mXM.K(ƣ3y<Re&s{awOF֜O Y'J<bUl$CBwtN>6{8f)t7
*%R?|0;5H8^$F"{LJKRJ*vz	Ǽ{d_B*7YLp-.Hid0&iQ Fdzs)	~%l<O$'$غN!qPXnQۊĀdWO8yZt^y3%p4.W\}-k܎	(q'aKq,<Q<.mSMsA䌨a
qx;Ubqb_F-
oIkE-OW5݋*@֌s6:伔}Vp1
help

Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?d96c7f42-f01b-8990-a558-ee92d631b51d>