From owner-freebsd-fs@freebsd.org Tue May 23 03:57:00 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 36AC7D79F56 for ; Tue, 23 May 2017 03:57:00 +0000 (UTC) (envelope-from aijazbaig1@gmail.com) Received: from mail-ua0-x22c.google.com (mail-ua0-x22c.google.com [IPv6:2607:f8b0:400c:c08::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EBF691A1C for ; Tue, 23 May 2017 03:56:59 +0000 (UTC) (envelope-from aijazbaig1@gmail.com) Received: by mail-ua0-x22c.google.com with SMTP id y4so64607500uay.2 for ; Mon, 22 May 2017 20:56:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=XNyrNGTnpVpEOq15SLV3kaUNN6oJO+bgMCHNWZanL2Y=; b=S43RmqtW1HQ5W7ZQD76zYlUL9c+BfaLB5JG7WcDEZY7OVVlGQh4ntLbOghKlkUn9mc OivL3Vj/H9ov3CQbbdOFacTJ7Y9P2PcXLQOt+nP5J54jBO5ZaJ4LuLM4BrQZ6SQdd5jg Eg3GsaTdcciGoejcV45FAY0/wux5qK9qF6yVaIaFCVHV99F1ijIy22XLxdi17y8HhTeO 5xMmA60zLrQbefdL4FuoaF8ECePzXMU2g3PAF86UOASSlLA/UgEOKMGaT93/AI8h2z4n KD2v5YRJ1vhUSOOOhrj1CmYty9uVnAzFdz2UkNnjNOYwj66IsC5KnO1WOt6br/2pydQs /dhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=XNyrNGTnpVpEOq15SLV3kaUNN6oJO+bgMCHNWZanL2Y=; b=ObaFs1JM1jdYg4WC+zczQ20bV0BHQLNJI95OB97Gg7asVHPOk9EsY2ZQx4YzaprRnP i15v63v+rpdAL9/DgDBtWxMFyHS8ntKNaDdZWKkzTKdr6VrMNBGaXPJp22UeI4o5gu8v 12N3xJ0QpapbzV8QzlgmHarOmL1zMyPV+FxgUVnR6Cytt6M4CssTmDtZhkeC4MXXzIMr jw7guxRlbkuVf5dhnPWxKDvdoNZex8iGzXheSAjjAZiTwg9Uj3a8frMZ/fh/y3Z6N5tQ whnc0p5ukzwibcj778NzBdEvpf/FSUN5ZwcDg/As/BIMXBbmiCXpJJoIuVE/StrxY/2l oVgA== X-Gm-Message-State: AODbwcDvNiztNfF3aD51jATQSxTu/9zp7S9bxsT3BTIBOtqIIyRFCrQ/ 1wSi+7bQhXazATNj8nOOv3zzapcKvkRx X-Received: by 10.176.16.235 with SMTP id x43mr14253863uab.15.1495511818700; Mon, 22 May 2017 20:56:58 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.204.195 with HTTP; Mon, 22 May 2017 20:56:58 -0700 (PDT) From: Aijaz Baig Date: Tue, 23 May 2017 09:26:58 +0530 Message-ID: Subject: Debugging nullfs - cannot access memory at address To: freebsd-fs@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 May 2017 03:57:00 -0000 I am trying to debug the nullfs kernel module so to that end, I do the following: On the target machine: kldstat gives Id Refs Address Size Name 1 10 0xffffffff80200000 17e10c8 kernel 2 1 0xffffffff819e2000 4cf0 vmxnet.ko 3 1 0xffffffff819e7000 16e0 echo.ko 4 1 0xffffffff81c11000 23dc vmmemctl.ko 5 1 0xffffffff81c14000 641b nullfs.ko nm /boot/kernel/nullfs.ko | grep mount 00000000000018f0 t null_getwritemount 0000000000000540 t nullfs_mount 0000000000000930 t nullfs_unmount U vfs_mountedfrom U vop_getwritemount_desc On the local machine (which connects to the target via a named pipe acting as a serial console (I am using virtual machines): (kgdb) tr0 kdb_sysctl_enter (oidp=, arg1=, arg2=0xfffffe004e7cc7f0, req=) at /usr/src/sys/kern/subr_kdb.c:446 446 kdb_why = KDB_WHY_UNSET; Current language: auto; currently minimal (kgdb) getsyms During symbol reading, Incomplete CFI data; unspecified registers at 0xffffffff8099497a. Id Refs Address Size Name 1 10 0x80200000 17e10c8 kernel 2 1 0x819e2000 4cf0 vmxnet.ko 3 1 0x819e7000 16e0 echo.ko 4 1 0x81c11000 23dc vmmemctl.ko 5 1 0x81c14000 641b nullfs.ko Select the list above with the mouse, paste into the screen and then press ^D. Yes, this is annoying. 5 1 0x81c14000 641b nullfs.ko add symbol table from file "/usr/obj/usr/src/sys/AIJAZ- DEBUG/modules/usr/src/sys/modules/nullfs/nullfs.ko.debug" at .text_addr = 0x81c14000 .data_addr = 0x81c14000 .bss_addr = 0x81c14000 (kgdb) add-kld nullfs.ko add symbol table from file "/boot/kernel/nullfs.ko.symbols" at .text_addr = 0xffffffff81c14000 set_sysinit_set_addr = 0xffffffff81c15c90 set_sysuninit_set_addr = 0xffffffff81c15cb0 .rodata.str1.1_addr = 0xffffffff81c15cc8 set_modmetadata_set_addr = 0xffffffff81c15e48 set_sysctl_set_addr = 0xffffffff81c15e58 .data_addr = 0xffffffff81c15e60 .bss_addr = 0xffffffff81c16360 (y or n) y Reading symbols from /boot/kernel/nullfs.ko.symbols... location expression too complex...done. (kgdb) b nullfs_mount Cannot access memory at address 0x81c14540 As one can see from the output of 'nm' and 'kldstat' above, the addresses are indeed proper. I even tried setting a "hardware breakpoint" at the above address (kgdb) hbreak *0x81c14540 Hardware assisted breakpoint 1 at 0x81c14540: file /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vfsops.c, line 74. (kgdb) c Continuing. Warning: Cannot insert breakpoint 1. Error accessing memory address 0x81c14540: Input/output error. On searching for this error on Linux, it appears that this is taken care of by turning off CONFIG_DEBUG_RODATA as part of the kernel config (which as per this:http://elinux.org/Overwrite_detection_for_ kernel_text_and_read-only_data link appears to be some sort of a protection mechanism which detects when the text section of the kernel is being overwritten for some reason). This helps with the setting of software breakpoints which would otherwise be not set. Is there something similar for FreeBSD systems as well?? Keen to hear -- Best Regards, Aijaz Baig