From owner-freebsd-embedded@freebsd.org Sat Sep 28 19:45:46 2019 Return-Path: Delivered-To: freebsd-embedded@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5C14812E83C for ; Sat, 28 Sep 2019 19:45:46 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene.sentex.ca (pyroxene.sentex.ca [IPv6:2607:f3e0:0:3::18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "pyroxene.sentex.ca", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46gfKQ1QCLz4WDK; Sat, 28 Sep 2019 19:45:46 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [192.168.43.26] (saphire3.sentex.ca [192.168.43.26]) by pyroxene.sentex.ca (8.15.2/8.15.2) with ESMTP id x8SJjiLl056416; Sat, 28 Sep 2019 15:45:44 -0400 (EDT) (envelope-from mike@sentex.net) Subject: Re: watchdogd stat location To: Ian Lepore , Warner Losh Cc: freebsd-embedded References: <5eba25eb-9ba4-0c93-27c8-e834491298ad@sentex.net> <83831ae6-9275-4f0c-a23d-c9cca3dc28f4@sentex.net> <817c7ed712d6b7da3015b7312be485a9044b14e1.camel@freebsd.org> From: Mike Tancsa Openpgp: preference=signencrypt Autocrypt: addr=mike@sentex.net; prefer-encrypt=mutual; keydata= mQENBEzcA24BCACpwI/iqOrs0GfQSfhA1v6Z8AcXVeGsRyKEKUpxoOYxXWc2z3vndbYlIP6E YJeifzKhS/9E+VjhhICaepLHfw865TDTUPr5D0Ed+edSsKjlnDtb6hfNJC00P7eoiuvi85TW F/gAxRY269A5d856bYrzLbkWp2lKUR3Bg6NnORtflGzx9ZWAltZbjYjjRqegPv0EQNYcHqWo eRpXilEo1ahT6nmOU8V7yEvT2j4wlLcQ6qg7w+N/vcBvyd/weiwHU+vTQ9mT61x5/wUrQhdw 2gJHeQXeDGMJV49RT2EEz+QVxaf477eyWsdQzPVjAKRMT3BVdK8WvpYAEfBAbXmkboOxABEB AAG0HG1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5jYT6JATgEEwECACIFAkzcA24CGwMGCwkI BwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEJXHwM2kc8rX+sMH/2V6pTBKsQ5mpWWLgs6wVP2k BC+6r/YKNXv9Rw/PrC6+9hTbgA+sSjJ+8gxsCbJsOQXZrxF0x3l9oYdYfuKcwdwXFX1/FS8p HfBeDkmlH+dI709xT9wgrR4dS5aMmKp0scPrXPIAKiYVOHjOlNItcLYTEEWEFBepheEVsgmk GrNbcrHwOx/u4igUQ8vcpyXPyUki+BsftPw8ZQvBU887igh0OxaCR8AurJppQ5UQd63r81cX E1ZjoFoWCaGK/SjPb/OhpYpu5swoZIhOxQbn7OtakYPsDd5t2A5KhvjI8BMTnd5Go+2xsCmr jlIEq8Bi29gCcfQUvNiClevi13ifmnm5AQ0ETNwDbgEIALWGNJHRAhpd0A4vtd3G0oRqMBcM FGThQr3qORmEBTPPEomTdBaHcn+Xl+3YUvTBD/67/mutWBwgp2R5gQOSqcM7axvgMSHbKqBL 9sd1LsLw0UT2O5AYxv3EwzhG84pwRg3XcUqvWA4lA8tIj/1q4Jzi5qOkg1zxq4W9qr9oiYK5 bBR638JUvr3eHMaz/Nz+sDVFgwHmXZj3M6aE5Ce9reCGbvrae7H5D5PPvtT3r22X8SqfVAiO TFKedCf/6jbSOedPN931FJQYopj9P6b3m0nI3ZiCDVSqeyOAIBLzm+RBUIU3brzoxDhYR8pz CJc2sK8l6YjqivPakrD86bFDff8AEQEAAYkBHwQYAQIACQUCTNwDbgIbDAAKCRCVx8DNpHPK 1+iQB/99aqNtez9ZTBWELj269La8ntuRx6gCpzfPXfn6SDIfTItDxTh1hrdRVP5QNGGF5wus N4EMwXouskva1hbFX3Pv72csYSxxEJXjW16oV8WK4KjKXoskLg2RyRP4uXqL7Mp2ezNtVY5F 9nu3fj4ydpHCSaqKy5xd70A8D50PfZsFgkrsa5gdQhPiGGEdxhq/XSeAAnZ4uVLJKarH+mj5 MEhgZPEBWkGrbDZpezl9qbFcUem/uT9x8FYT/JIztMVh9qDcdP5tzANW5J7nvgXjska+VFGY ryZK4SPDczh74mn6GI/+RBi7OUzXXPgpPBrhS5FByjwCqjjsSpTjTds+NGIY Organization: Sentex Communications Message-ID: Date: Sat, 28 Sep 2019 15:45:43 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: <817c7ed712d6b7da3015b7312be485a9044b14e1.camel@freebsd.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 46gfKQ1QCLz4WDK X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-5.99 / 15.00]; NEURAL_HAM_MEDIUM(-0.99)[-0.991,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2019 19:45:46 -0000 On 9/28/2019 3:30 PM, Ian Lepore wrote: > If we want to be sure to force physical IO, how about dd if=/ > of=/dev/null count=1 ? > > But I question the premise of forcing physical IO as being somehow a > better indicator of a non-hung system. I think it's just a better > indicator of the sdcard problem that Mike is experiencing. For anyone > else, forcing periodic physical IO is going to do annoying things like > spin up idle drives. I think in my case, I am going to need to do that.  I was hoping doing a simple stat on / or /boot would do the trick to recover from mmcsd0: Error indicated: 1 Timeout g_vfs_done():mmcsd0s1a[READ(offset=267358208, length=4096)]error = 5 vnode_pager_generic_getpages_done: I/O read error 5 vm_fault: pager read error, pid 1 (init) sdhci_pci0-slot0: Got AutoCMD12 error 0x0001, but there is no active command. sdhci_pci0-slot0: ============== REGISTER DUMP ============== sdhci_pci0-slot0: Sys addr: 0x74ee0000 | Version:  0x00001001 sdhci_pci0-slot0: Blk size: 0x00005200 | Blk cnt:  0x00000008 sdhci_pci0-slot0: Argument: 0x0007f817 | Trn mode: 0x00000037 sdhci_pci0-slot0: Present:  0x01ff0000 | Host ctl: 0x00000007 sdhci_pci0-slot0: Power:    0x0000000f | Blk gap:  0x00000000 sdhci_pci0-slot0: Wake-up:  0x00000000 | Clock:    0x00000007 sdhci_pci0-slot0: Timeout:  0x0000000d | Int stat: 0x00000000 sdhci_pci0-slot0: Int enab: 0x01ff00fb | Sig enab: 0x01ff00fb sdhci_pci0-slot0: AC12 err: 0x00000001 | Host ctl2:0x00000080 sdhci_pci0-slot0: Caps:     0x21fe32b2 | Caps2:    0x00000070 sdhci_pci0-slot0: Max curr: 0x00c80064 | ADMA err: 0x00000000 sdhci_pci0-slot0: ADMA addr:0x00000000 | Slot int: 0x000000ff sdhci_pci0-slot0: =========================================== g_vfs_done():mmcsd0s1a[READ(offset=267358208, length=4096)]error = 5 vnode_pager_generic_getpages_done: I/O read error 5 but it looks like no dice, at least in the one case I hit over the weekend. However from the captured logs, not sure if watchogd really got armed or not. I think doing an actual raw read is the way to go, but to put that in watchdogd feels like it would violate POLA.  I think instead, I will make it an external command as it will fix my needs, or even roll my own watchdogd which might even be better for me.     ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 x203 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada