From owner-freebsd-hackers@freebsd.org Sun Mar 28 00:03:41 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5EF315ACDA2 for ; Sun, 28 Mar 2021 00:03:41 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qk1-x729.google.com (mail-qk1-x729.google.com [IPv6:2607:f8b0:4864:20::729]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F7GC02Xvhz4sBm for ; Sun, 28 Mar 2021 00:03:40 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-qk1-x729.google.com with SMTP id q3so9030949qkq.12 for ; Sat, 27 Mar 2021 17:03:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:subject:message-id:mime-version :content-disposition; bh=MsWly9YKpnslp1L7NbZQaoCylYRMfrgxIHQKG7InyMs=; b=tET/RcImrciQbrZ9/m8/UwV9Zwp8elLYcluPqXf/IbCHTnd+n2gSnbiwJxsRXzw0La 77XV7q6Svhq5+AcljihVgGSLTBTLKcSzvvBqeay8Ghd/RcFVEYcyLrKUrVQ9WOeHFN18 FDX2m9nfwt8ZlwWY1+ZZSKHjnT6GaGjGwXp9gyjJShVvbbfd0Hi0b0tSGNDcxbV+WOH4 c6QeAtiQ5PmpWXP7VpCr09WSx3gCq6aK38xOkHsBNX6CNKK/XVWwnj+wSmhJt7QZN/n1 AyaXPX9tkIy6oLGbHN/3rEiKI3iukAGm0LWkBwIy3wLrZQW1MZY3eh4h8btnbEd/Bpr7 rtdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:subject:message-id :mime-version:content-disposition; bh=MsWly9YKpnslp1L7NbZQaoCylYRMfrgxIHQKG7InyMs=; b=icbLAJM67OAx1Th9km7vGhi8zwnK5X6g+llIDtL6VmSPvQA8TsMHeqrDbkBaTAPpfR kuCK3Ir+u65viAEpT39AwOAFsAge3MyTQb8RpKA5NkVwTUn8QAZUBtqpQU9+GvhpKKbm oUGf6Botr7XbxHjMGdQ6WmI0hwezBFz48XPZEi8AJh2SpPbFWLtU8OokyrxDsTyNkFqI D6LUw6h008bUNPgpqXiUCa3YOJdEAW5BD2GLvxqeXWeKwgDelx3Sctniv/bpmaxIyLlO JBdtJIqB10nL7PWOY9Mya0mE7nJiRVGDOsQrVSbxsQBXig9YBe6o5m/1Gtag2k7TyPXp AYRA== X-Gm-Message-State: AOAM531KGdpRKNpAvdxZ8h/SLpSRukayoqscKmUzWrMUIEVyavwbrMrk Gae2pJ4Ep6su9RQT+iJ2WPJUDVV4W4rfWg== X-Google-Smtp-Source: ABdhPJxZf/H2Jg4SaSx4kdCAGp2aYvJG2uIIKIj0RK+YIrewueaIZXBlZD8s2KKFD9zl1HvKTduIdA== X-Received: by 2002:a37:c13:: with SMTP id 19mr19111639qkm.210.1616889819275; Sat, 27 Mar 2021 17:03:39 -0700 (PDT) Received: from nuc ([142.126.164.150]) by smtp.gmail.com with ESMTPSA id w78sm10019536qkb.11.2021.03.27.17.03.38 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 27 Mar 2021 17:03:38 -0700 (PDT) Sender: Mark Johnston Date: Sat, 27 Mar 2021 20:03:39 -0400 From: Mark Johnston To: freebsd-hackers@freebsd.org Subject: KASAN port Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Rspamd-Queue-Id: 4F7GC02Xvhz4sBm X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=tET/RcIm; dmarc=none; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::729 as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-2.69 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; NEURAL_HAM_SHORT(-0.99)[-0.995]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::729:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; TO_DOM_EQ_FROM_DOM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::729:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::729:from]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Mar 2021 00:03:41 -0000 Hi, I ported the KASAN implementation from NetBSD to FreeBSD. This is a testing and debugging tool that leverages compiler instrumentation to maintain a kernel "shadow" map which stores information about which addresses in the main kernel are safe to access. If you've been paying attention to recent kernel commits you may have noticed that several bugs have been found and fixed using this tool already; there are several more that I'm aiming to have fixed in 13.0. There was a GSOC project by Costin Carabas and andrew@ which did an initial port of KASAN and several other debugging facilities; I reused a few pieces of that work but this was mostly a direct port. The instrumentation and validity checking introduces a fairly substantial performance hit. I think a 2-3x slowdown is pretty typical, but it could be more for workloads which execute a lot of kernel code. It's best used in conjunction with test suites that exercise a lot of kernel functionality, like the regression test suite, stress2 or syzkaller. KASAN is currently only implemented for amd64. It would be a useful and probably relatively small project to port it to platforms like arm64 and riscv. If anyone is interested in this, please contact me. I posted reviews for various pieces of the port here: https://reviews.freebsd.org/D29454: Add a KASAN option to the kernel build https://reviews.freebsd.org/D29416: Add the KASAN runtime https://reviews.freebsd.org/D29417: amd64: Implement a KASAN shadow map https://reviews.freebsd.org/D29455: amd64: Add MD bits for KASAN https://reviews.freebsd.org/D29456: uma: Add KASAN state transitions https://reviews.freebsd.org/D29457: kstack: Add KASAN state transitions https://reviews.freebsd.org/D29458: kmem: Add KASAN state transitions https://reviews.freebsd.org/D29459: vfs: Add KASAN state transitions for vnodes https://reviews.freebsd.org/D29460: execve: Mark exec argument buffers https://reviews.freebsd.org/D29461: malloc: Add state transitions for KASAN A couple of small LLVM changes are also required: https://reviews.llvm.org/D98285 https://reviews.llvm.org/D98286 Please ask questions and provide review feedback. To test the port, grab https://github.com/markjdb/freebsd/tree/ff/kasan and: $ make kernel-toolchain WITHOUT_SYSTEM_COMPILER= $ make buildkernel KERNCONF=GENERIC-KASAN There are a few limitations of the current implementation, especially from the fact that we don't have a way to disable all uses of the direct map. However, we have a way to reduce usage of that map by kernel memory allocators and that's enough to find non-trivial bugs, so it seems worthwhile to commit it now and continue to refine it.