From owner-freebsd-hackers@freebsd.org Sun Oct 16 20:39:33 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DEF0AC145CA for ; Sun, 16 Oct 2016 20:39:33 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-io0-x232.google.com (mail-io0-x232.google.com [IPv6:2607:f8b0:4001:c06::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AB6551CBF for ; Sun, 16 Oct 2016 20:39:33 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: by mail-io0-x232.google.com with SMTP id i202so169175293ioi.2 for ; Sun, 16 Oct 2016 13:39:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=uwCS5IEki76kquntpy4lRv5Y9hIYLtNK3wtuxHSEfcM=; b=CgOSWxYrk+KsqcK5UfZryqn2pIrVorHc6emznGH/dhhmB+rhJ+QUYiUM4ZlSJY92tc 9bP88ngGccrv568A7iM2pqFK6QnWP+4+MX9bgLSKOybLzw5cmMYc9q5MLYOQq4lC14CK aovLdHSUoYfbDeM1aliae4lP4fzzeoROTrvC9uRa9DTRILQhoA2vyU5sPFAOJnnZxrwn m253J0EhFjuJEbzJT0YYKKhdXDZXFOU0nVwrionDb1B2xW76HZwwqARbv/miSZmprC67 x3B3FTBRKbUbasCBrp3kTFt7Lt98rhnUE6rlV8NpoCygq+Vzf8EmEWbr982I6PGFhT/C 1huQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=uwCS5IEki76kquntpy4lRv5Y9hIYLtNK3wtuxHSEfcM=; b=EB/6C+kHxS+lbuF5kEc1jKE1+4tnCb4hXNBBl2L97nZ7JiueOHJqLbRmXbwtz8RhNt DjfrOug89thGoBHukkopba94x60T55IIUJIWxCey34qCCx7M+GI+e5MIcjfWKqptbwIx /KCCJoZ5KCGgeZwkO/o7DIPwMLy+NoRWaxgJkjikZs6Gs2BxVQolrs4uwvEsYmHJ95/5 09QIQSWqVtrrdRbQCOoj1IJbTdQ0xp30KNikUrFNiyEzEntmNOOOOmcVluje3bM+7pzb PzMdHK93vS+QumRqQc/9TKExYL8iFKcdziNm09iejmhveOHUHpEjq6Zrd9gpYyFGC+T+ Y6RA== X-Gm-Message-State: AA6/9RmFmu+xMq6M951GJt5Tayw4AW6SSQXlUSIzDdldjCmUN8DshGLCE95oVyw9hw0XrcSfLdoz9YwfFUl1mw== X-Received: by 10.107.17.229 with SMTP id 98mr19507945ior.96.1476650372884; Sun, 16 Oct 2016 13:39:32 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.7.129 with HTTP; Sun, 16 Oct 2016 13:39:32 -0700 (PDT) From: Aryeh Friedman Date: Sun, 16 Oct 2016 16:39:32 -0400 Message-ID: Subject: why doesn't root-on-ZFS mount bootpool by default? To: FreeBSD Mailing List Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 20:39:34 -0000 It is very annoying (make world and install kmods from ports) to not have /bootpool be mounted by default (I found the right fixes [not documented in the handbook])... why is this the case? -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-hackers@freebsd.org Sun Oct 16 22:41:15 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06AE1C14FF4 for ; Sun, 16 Oct 2016 22:41:15 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-lf0-x22f.google.com (mail-lf0-x22f.google.com [IPv6:2a00:1450:4010:c07::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 81EE91159 for ; Sun, 16 Oct 2016 22:41:14 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-lf0-x22f.google.com with SMTP id l131so212316900lfl.2 for ; Sun, 16 Oct 2016 15:41:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=nW9sNvQnsHyRlUoqFRqTdoLQQY8Y6iAYadrZrltTbdA=; b=k2r3+X9cBtuRoHanwDJ0uvzJxvEi0W7S9a/B/7fKSEdU4nBlJuZ1qENRlPl2sNzl9J ZbJSa1+oXwI06oTBu7Pj4QnrgRCuB+YKW31clG20MStXajft/pU9e5vUkGMxUe+A82ks zm3Tb7vBAAGhIu/lTr+GrhUWM34t6MmMwtLqSK2qxZyuvTlhAHqEhlkoqYCXDFeMu1qX kWNaOcJ7XwqnBIrJmsD+eJ5NQAwXRBbWlCD0Q5cqzzGlCF/mXU4ScZH2ES49LNB7qk+8 ExOdQbDPERdJuBzK1TuygfmuBg6ecYxscOF5ZtW/DVCouB9M0Q4WJu18RvM5BlQC7ZT1 CHFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=nW9sNvQnsHyRlUoqFRqTdoLQQY8Y6iAYadrZrltTbdA=; b=cKwW/I1pUTMU+PtgTuIvGu2SAcRyS14cSnDW7O2qg95t8dttTMoAgX+MWabFIJxEha RLtWHDPeSvlBL6A4uzlru/6Bfz4rdMI2JVlABQJNiY5xMwwJESrh95vr+vOwReBOOO3x 6fOkSgIXC5wY/objsB425vxNKxG6HAaZXUa0llIKd69V+Izl60cRMhhQS6uSR8eMU/U2 FH1fMKXpKYv0nEKTw04ik/FUcUDhvGt2mYxvByWk28Of4aOoW0aZJXQdxnm5cWjs7lYX 6yoLzbFOlRh09t1eZ4nmL3eZnqy7LTeCjoLF5HQJnxwHwSbzrEJq3q37QybASG8dY0f3 N9pA== X-Gm-Message-State: AA6/9Rl4ykNR4QsQz3UhwwEfdeBYfesk6xVOZF11nz+K2eIfjBGQJd65G2U61ZVHdaMYBfZE X-Received: by 10.28.126.21 with SMTP id z21mr5909418wmc.71.1476657672265; Sun, 16 Oct 2016 15:41:12 -0700 (PDT) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id f2sm46743185wjr.2.2016.10.16.15.41.11 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 16 Oct 2016 15:41:11 -0700 (PDT) Subject: Re: why doesn't root-on-ZFS mount bootpool by default? To: freebsd-hackers@freebsd.org References: From: Steven Hartland Message-ID: <0ec9b6bc-a2a7-772f-fb84-fac5ed3b71df@multiplay.co.uk> Date: Sun, 16 Oct 2016 23:41:37 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 22:41:15 -0000 We'll need some more details to go on before your question can be answered properly. Please post all the relevant details about your setup, what you did, what you expected and what actually happened? On 16/10/2016 21:39, Aryeh Friedman wrote: > It is very annoying (make world and install kmods from ports) to not have > /bootpool be mounted by default (I found the right fixes [not documented in > the handbook])... why is this the case? > From owner-freebsd-hackers@freebsd.org Sun Oct 16 22:46:50 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 86345C14289 for ; Sun, 16 Oct 2016 22:46:50 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-io0-x235.google.com (mail-io0-x235.google.com [IPv6:2607:f8b0:4001:c06::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4EEF415A0 for ; Sun, 16 Oct 2016 22:46:50 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: by mail-io0-x235.google.com with SMTP id j37so171535125ioo.3 for ; Sun, 16 Oct 2016 15:46:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=6EeZYwES+CJb/9YifRtOY0/UBHUanPLP77tqTbmXiEw=; b=ZNYK/4WJV6uFqBYzRCenJibHf+jN1SQSoNjHkhb8j4XtDImWjE3uvxFtprRDuJ9XdZ P/EUOayVHhN18RGiiP3pb0nY4GTKVy3NDiN8MW6lBaGg1pGXpjL6veRjr08YUfrB8BU6 M9fiYhe4Wh0oWZTmnMNGrQK2kInjf5a/VQJ0B62/U4FuV7a9QwKx5NOR0idr2lbPsgjo ij9LSfDS0+pvG2R8FipHlASsOzm2EGQZMSJNwrwtzaCMLBerMhXrWTScMELluWzwNnmL VoencSq7q0aHNi9cZ2DYtZ9WtN3e0UEAQpwA//GjP/w81/Di5OC8NCM60t5j+HZ+bKkZ N3nA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=6EeZYwES+CJb/9YifRtOY0/UBHUanPLP77tqTbmXiEw=; b=KXyQ7s96ynvlZvRF/AymnLVqGegG6OES7Lh90zjveWZNSvKDmHorr73sAcEMKsbzLF zVK6cK+5stYXtdjrEGYuNn9PE5GF9xrCd8v8EKAG1/3pdR6RSK3DLzJggf8w1dO5jJ2x N1bcXzd7UMFDCLb7SGkdVWlDKteqjGqJIRIHDvSe3qNqYFc0oc+FrBcpjilqn4sHlika /HPexwoqYI7Q9B9WAw3pJICYFcthRFgrRoXJTN409HH8aHijlr9GjV9eTX80fmK8ueO5 +2E/4pQS7HsrsUDJ3hMQH5BnXStZdx/4d7Y/7l/J4Zd30M1Eeh9dOwbKDzuv9jDVULYc jhMQ== X-Gm-Message-State: AA6/9RmjETudXouuJ/PlPATWOZjCb/8bZk+W8U1mhGIKXRJVTfxxULuQlL7fpZ2ILhOi9VTF1PP/xnWwwkKxEw== X-Received: by 10.107.172.133 with SMTP id v127mr2949270ioe.49.1476658009704; Sun, 16 Oct 2016 15:46:49 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.7.129 with HTTP; Sun, 16 Oct 2016 15:46:49 -0700 (PDT) In-Reply-To: <0ec9b6bc-a2a7-772f-fb84-fac5ed3b71df@multiplay.co.uk> References: <0ec9b6bc-a2a7-772f-fb84-fac5ed3b71df@multiplay.co.uk> From: Aryeh Friedman Date: Sun, 16 Oct 2016 18:46:49 -0400 Message-ID: Subject: Re: why doesn't root-on-ZFS mount bootpool by default? To: Steven Hartland Cc: FreeBSD Mailing List Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 22:46:50 -0000 Fresh install of 11.0-PL1 with MBD/Stripped (single disk) non-encrypted has the following mount points: zroot/ROOT/default 933311872 1177108 932134764 0% / devfs 1 1 0 100% /dev bootpool 2015288 123760 1891528 6% /bootpool zroot/tmp 932134860 96 932134764 0% /tmp zroot/usr/home 932134900 136 932134764 0% /usr/home zroot/usr/ports 932134860 96 932134764 0% /usr/ports zroot/usr/src 933363924 1229160 932134764 0% /usr/src zroot/var/audit 932134860 96 932134764 0% /var/audit zroot/var/crash 932134860 96 932134764 0% /var/crash zroot/var/log 932134916 152 932134764 0% /var/log zroot/var/mail 932134860 96 932134764 0% /var/mail zroot/var/tmp 932134860 96 932134764 0% /var/tmp zroot 932134860 96 932134764 0% /zroot If fixed with zfs import -f bootpool and then adding the following to loader.conf: zpool_cache_load="YES" zpool_cache_type="/boot/zfs/zpool.cache" zpool_cache_name="/boot/zfs/zpool.cache" On Sun, Oct 16, 2016 at 6:41 PM, Steven Hartland wrote: > We'll need some more details to go on before your question can be answered > properly. > > Please post all the relevant details about your setup, what you did, what > you expected and what actually happened? > > On 16/10/2016 21:39, Aryeh Friedman wrote: > >> It is very annoying (make world and install kmods from ports) to not have >> /bootpool be mounted by default (I found the right fixes [not documented >> in >> the handbook])... why is this the case? >> >> > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-hackers@freebsd.org Sun Oct 16 22:47:27 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 42DC4C14343 for ; Sun, 16 Oct 2016 22:47:27 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-it0-x229.google.com (mail-it0-x229.google.com [IPv6:2607:f8b0:4001:c0b::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0A1D716BB for ; Sun, 16 Oct 2016 22:47:27 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: by mail-it0-x229.google.com with SMTP id 4so35698749itv.0 for ; Sun, 16 Oct 2016 15:47:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=U6LvZLtozg4j7y5xHNEyVJOCf7xSen8Fw6IwOHi/gdk=; b=QzX7rXmbhT8RdwJ5qbtrzeUCZ2cYektBC1KluJveTr0o2yRcDTFKC70wrCfBjIbghP 8pxxWfom9Jvg7HP6BqXamyS5LRHIZJSTqJMcZskJgGN0/icFEHVSBr5nEgoS8xIL9UMB 9ZSfUDPYwypxMsFIpGvX7MguQBOk0Q9anDU8p6RgIcvpW+qiH1+8ctFTfiKY45iTexwH f/We9MHVq8xuzQ0FjbxaiNPtykRB8D8MLz7cyNl+KwbytQvJbuwYsZTdDv4XlzCdjQMZ 6un56KbQTXS21ckzpyYLoQGcKHY5fLC0mhDcnvD/EHHxU1bbiRGip66EbEnNbcXpMXkh eWXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=U6LvZLtozg4j7y5xHNEyVJOCf7xSen8Fw6IwOHi/gdk=; b=Hxho0O0d3i0ori9u/I8o22sdb66vCEk1UEba93ZmbiCl+DaTB39e79ogHJijYdvVEC PS3Vo6O+IdxXR70QayXPMDYrDWh9/PeviOliBpCaU6hNkuMAXpBYKzJbPS4yBfWfEAbR E5Zh8Q49Ra4Bsd/RAd1KIXIbeU+wOANyySYlfhV78d9/NiTF+MxHS/SyT5Gjj0mQEmim XOi2bRw7CUPbnVLP0LgvFeJmNyngzNRZvIU9x9k75jLQmnvJiLDERdgGCc5DgcvHdByY RFyfBtTpoqK4dAFsYyoIxMG566IaRpKGs0D511xye4StqxoPFxvO0FJjZzyUrfBnM35S ckqg== X-Gm-Message-State: AA6/9RmYjz2h6YsUwHdtAIWGKf9o7P/nOPt2tqWiYm3zOf0DjjIbY7DbOy+YNBZc2ApE2dcKS7jRpzm6q0Q01g== X-Received: by 10.36.207.66 with SMTP id y63mr6249836itf.105.1476658046523; Sun, 16 Oct 2016 15:47:26 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.7.129 with HTTP; Sun, 16 Oct 2016 15:47:26 -0700 (PDT) In-Reply-To: References: <0ec9b6bc-a2a7-772f-fb84-fac5ed3b71df@multiplay.co.uk> From: Aryeh Friedman Date: Sun, 16 Oct 2016 18:47:26 -0400 Message-ID: Subject: Re: why doesn't root-on-ZFS mount bootpool by default? To: Steven Hartland Cc: FreeBSD Mailing List Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 22:47:27 -0000 oops /bootpool is not on the default list On Sun, Oct 16, 2016 at 6:46 PM, Aryeh Friedman wrote: > Fresh install of 11.0-PL1 with MBD/Stripped (single disk) non-encrypted > has the following mount points: > > zroot/ROOT/default 933311872 1177108 932134764 0% / > devfs 1 1 0 100% /dev > bootpool 2015288 123760 1891528 6% /bootpool > zroot/tmp 932134860 96 932134764 0% /tmp > zroot/usr/home 932134900 136 932134764 0% /usr/home > zroot/usr/ports 932134860 96 932134764 0% /usr/ports > zroot/usr/src 933363924 1229160 932134764 0% /usr/src > zroot/var/audit 932134860 96 932134764 0% /var/audit > zroot/var/crash 932134860 96 932134764 0% /var/crash > zroot/var/log 932134916 152 932134764 0% /var/log > zroot/var/mail 932134860 96 932134764 0% /var/mail > zroot/var/tmp 932134860 96 932134764 0% /var/tmp > zroot 932134860 96 932134764 0% /zroot > > If fixed with zfs import -f bootpool and then adding the following to > loader.conf: > > zpool_cache_load="YES" > zpool_cache_type="/boot/zfs/zpool.cache" > zpool_cache_name="/boot/zfs/zpool.cache" > > > > On Sun, Oct 16, 2016 at 6:41 PM, Steven Hartland > wrote: > >> We'll need some more details to go on before your question can be >> answered properly. >> >> Please post all the relevant details about your setup, what you did, what >> you expected and what actually happened? >> >> On 16/10/2016 21:39, Aryeh Friedman wrote: >> >>> It is very annoying (make world and install kmods from ports) to not have >>> /bootpool be mounted by default (I found the right fixes [not documented >>> in >>> the handbook])... why is this the case? >>> >>> >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org >> " >> > > > > -- > Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org > -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-hackers@freebsd.org Mon Oct 17 13:41:46 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C985DC12BBE for ; Mon, 17 Oct 2016 13:41:46 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (smtp.digiware.nl [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 92245D31; Mon, 17 Oct 2016 13:41:46 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from router.digiware.nl (localhost.digiware.nl [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 806E72A08D; Mon, 17 Oct 2016 15:41:42 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.com Received: from smtp.digiware.nl ([127.0.0.1]) by router.digiware.nl (router.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gJAvDMcABS-w; Mon, 17 Oct 2016 15:41:41 +0200 (CEST) Received: from [192.168.10.67] (opteron [192.168.10.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id AD1892A089; Mon, 17 Oct 2016 15:41:41 +0200 (CEST) Subject: Re: Kqueue and threading To: Eric Badger , freebsd-hackers@freebsd.org References: <111e0c35-7a4b-b6c7-ef1d-1a0d85112e61@digiware.nl> <4209b8d4-6674-a51d-dceb-81c3ecd179c2@FreeBSD.org> From: Willem Jan Withagen Message-ID: <6441cf69-f559-fe5a-fe9e-464b0f13c3b4@digiware.nl> Date: Mon, 17 Oct 2016 15:41:39 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <4209b8d4-6674-a51d-dceb-81c3ecd179c2@FreeBSD.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 13:41:46 -0000 On 3-10-2016 19:48, Eric Badger wrote: > On 10/01/2016 13:02, Willem Jan Withagen wrote: >> Hi, >> >> Ceph uses a lot of threading, and for any part of it communication it >> uses epoll(), which FreeBSD does not use. For that there was already a >> EvenKqueue implementation. >> >> But I think I'm now running into: >> The kqueue() system call creates a new kernel event queue and >> returns a descriptor. The queue is not inherited by a child >> created with fork(2). However, if rfork(2) is called without the >> RFFDG flag, then the descriptor table is shared, which will allow >> sharing of the kqueue between two processes. >> >> Kqueue descriptors are created and events are added, but then the >> process starts other threads and expects the kqueue-id to be valid there. >> >> However adding more events returns an error, also waiting on the ID for >> events to happen returns an error (descriptor invalid) >> >> Threading is done with 2 different constructions: >> std::thread >> and creating Workers >> >> Would any of these qualify with the quoted warning? and invalidate the >> descriptor table? >> >> If not, how can I (easily) debug the reason why my descriptors go invalid? >> > > Sharing a kqueue between threads of a process works. Are the workers > created using rfork without RFFDG as suggested in the manpage? I've > never had reason to do this, but a quick test seems to indicate that it > works as advertised. A normal fork closes the kqueue file descriptor. If > you suspect that's what's happening, you might run "procstat -f {worker > pid}" to see if file descriptors with a "k" (kqueue) in the "T" (type) > column appear (if not, they were presumably closed on the fork). Right, This has become a rather serious and painful undertaking, but it seems to work now, and all tests seem to be working oke .... But getting at the workers creation is rather impossible since that is mostly hidden in some Boost libs... Fortunate the systemcode for events is rather fleshed out into modules and there is a EventKqueue class, that hides all gory details. So there I main lists of what events are being set, and once I detect a change of thread, I reinstall the list with events. But that is obviously rather cumbersome. --WjW So what I do know From owner-freebsd-hackers@freebsd.org Wed Oct 19 07:49:34 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C1E4C17B0C for ; Wed, 19 Oct 2016 07:49:34 +0000 (UTC) (envelope-from damian@damianek.be) Received: from mail-io0-x232.google.com (mail-io0-x232.google.com [IPv6:2607:f8b0:4001:c06::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DFE81F4 for ; Wed, 19 Oct 2016 07:49:33 +0000 (UTC) (envelope-from damian@damianek.be) Received: by mail-io0-x232.google.com with SMTP id f5so23282033ioj.2 for ; Wed, 19 Oct 2016 00:49:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=damianek-be.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=owYTJI9PZqrKuRkl8HR5uocfM+L1e3COez16AdqP7Uw=; b=T5JnEMkjzugrbUwWsKoMY3Y6ckNKNc2fOtbXb3qXHgTR/g2p+f9la9UnAIA0IH/3NE 9Up6XTtzgN0sgRY1vkcZ1AKqnWEuYBdIL20xxSPn8nBjrspORmcDtUj0iA1PmwoXowrH Jv168O3g/Um7I3V5ed/L+1EvMvQSOlXAywl3DGUUom6PkNR+UanTwmArReq5qTlj1nJh poz/xwI2Z63TMMS1IFU5nMPAxBkHcsae9AU/snz1WLKvQGvAuZ9F4oKwLDLarfhy95c1 vvrNs1PaphyRTttSTwMEWQoJh7bQPVysh5vvPIZ7tytAtMe5gXrY7BflBpGehn+4haFS 37ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=owYTJI9PZqrKuRkl8HR5uocfM+L1e3COez16AdqP7Uw=; b=E+H+rzY3yhQd3imWmKzMbjBe+46//gik5Ch4vBM/egNmfMDxB9b0nxplKLE4w5fDEh vyrnE5Q+CjKojb9odMY33Q1btGNHcZKRDtMCzxu+epefEkLx3ix2Vzd1cmYvoF1L/ayi kdxgVvpcF1Zj1Sz8dP/iCT6zSGp+cO1GxaAmvpF7QFwn6Dm2RC5ssIjfWCZpvE51jonj bQJSVPDIDZCppEU6K9EFiFwSarknr4XdvvnsFGJbUczcwS6Z1fuBOeW4IMJXzdkmWrub 1G7U6gDUxq0MDBElDuLoD84gzz0a2yhHSqkO+AOgWwR2+y4rVPR2pbwMddm3nC3D6pJT Gz5Q== X-Gm-Message-State: AA6/9Rn+kZSJICYTL966N1SSTyisqGRdmzfHVjeD4UbNsnmzFnB08GhS2rujW6RdIqFk4J0oWjHDIkZALqY1kg== X-Received: by 10.107.154.134 with SMTP id c128mr5846684ioe.184.1476863372024; Wed, 19 Oct 2016 00:49:32 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.39.71 with HTTP; Wed, 19 Oct 2016 00:48:51 -0700 (PDT) From: "damian@damianek.be" Date: Wed, 19 Oct 2016 09:48:51 +0200 Message-ID: Subject: FreeBSD11 buildworld WITHOUT_ICONV To: freebsd-hackers@freebsd.org X-Mailman-Approved-At: Wed, 19 Oct 2016 10:59:21 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 07:49:34 -0000 Hello. FreeBSD 10.3-STABLE amd64 1003508 In my /etc/src.conf i have 'WITHOUT_ICONV=yes' and i try buildworld FreeBSD 11.0-STABLE, but: ===> lib/libc/tests/iconv (all) (cd /usr/src/lib/libc/tests/iconv && DEPENDFILE=.depend.iconvctl_test NO_SUBDIR=1 /usr/obj/usr/src/make.amd64/bmake -f /usr/src/lib/libc/tests/iconv/Makefile _RECURSING_PROGS=t PROG=iconvctl_test ) echo iconvctl_test.full: /usr/obj/usr/src/tmp/usr/lib/libc.a /usr/obj/usr/src/tmp/usr/lib/libprivateatf-c.a >> .depend.iconvctl_test cc -O2 -pipe -march=native -g -MD -MF.depend.iconvctl_test.iconvctl_test.o -MTiconvctl_test.o -std=gnu99 -fstack-protector-strong -Qunused-arguments -c /usr/src/lib/libc/tests/iconv/iconvctl_test.c -o iconvctl_test.o /usr/src/lib/libc/tests/iconv/iconvctl_test.c:30:10: fatal error: 'iconv.h' file not found #include ^ 1 error generated. *** Error code 1 WITHOUT_ICONV option is respected and iconv is not compiled, but in "stage 4.3: building everything" still this iconv is needed. -- damian@damianek.be From owner-freebsd-hackers@freebsd.org Wed Oct 19 17:51:50 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 30AEBC19257 for ; Wed, 19 Oct 2016 17:51:50 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pf0-x235.google.com (mail-pf0-x235.google.com [IPv6:2607:f8b0:400e:c00::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 00F30C6E for ; Wed, 19 Oct 2016 17:51:50 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mail-pf0-x235.google.com with SMTP id e6so19758900pfk.3 for ; Wed, 19 Oct 2016 10:51:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=iCMZpAfoRKggzPBL3BrweHc5G8Kyy3lqAoRXT49QD1g=; b=SmZ3uP9GL78fwYiShns6z6MP9hSdnhqtgW2vf7q4wZvo5ssb2oFZjM4UTLIVis7QiA 8bSjc6SfsvKjOqxLNbBmkYGNlyD7k0ZW/TcFSKQ6NdMFSW4H4o6Gpvm9Zfcc3qlA1ZpE QNgKLA8tGdb4Ul97n2IU6oaow7zBAPiQwomgV9QkYRev4UAC9kccAU6EzgmrJOi21j9q Sh3cxmDphtNxlkQl+EorCaiHGGnvbFfn8wijkH77q/t2Ds8g9g4s6SeQPpLj6XqL/xD7 K5GnR0kxrgpOwY/SnSnHTGxo1awYrps+rbQJ+WQO9Qgyw2steVXzdsVG5U8tDDvI8tG1 vwFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=iCMZpAfoRKggzPBL3BrweHc5G8Kyy3lqAoRXT49QD1g=; b=l5PG/l08kqj2xfBusVJd4doBkBqRoBmVjYLK2PMobINv5DoIWzoIw/i0vxbxT8cAIv UXHk/Sk4Oj+7f3RaUl5vjiqX741f33SpjCYFCfoImO+DMcoikEm/91cD7J+yPn0u70os Gv7p2s+S6sEHhSqhMaPBHi/vf7fAwpyKiwpp1gffVapy2hoJbEKs4tnqaqyyB2aBcciI EQ0yTd/APC1Ks60tnZ7LrDb6q2cuu1RwD9ijTIb89thBEX1nAcPy4TIdO2REwJDKjpzC Jv2WeaQy6j17yllfz98d+g8veeS7BMRH343lPJVO0iic9/srv4NKiKm3mEhU5AL+6Caz gmSA== X-Gm-Message-State: AA6/9Rnd//WYhe27IZOBajmWzQNlU+G7Jm0GzH8S1BMIVLZ9fLDZa+D5naBQdSyKHjWMwQ== X-Received: by 10.99.114.80 with SMTP id c16mr5436587pgn.144.1476899509135; Wed, 19 Oct 2016 10:51:49 -0700 (PDT) Received: from ?IPv6:2607:fb90:80bb:5bf8:8135:9c53:abb9:3522? ([2607:fb90:80bb:5bf8:8135:9c53:abb9:3522]) by smtp.gmail.com with ESMTPSA id x190sm65434319pfd.20.2016.10.19.10.51.48 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 19 Oct 2016 10:51:48 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: FreeBSD11 buildworld WITHOUT_ICONV From: Ngie Cooper X-Mailer: iPhone Mail (14A456) In-Reply-To: Date: Wed, 19 Oct 2016 10:51:47 -0700 Cc: freebsd-hackers@freebsd.org Content-Transfer-Encoding: 7bit Message-Id: <30603328-0967-4F43-AF8F-1291439DEEE3@gmail.com> References: To: "damian@damianek.be" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 17:51:50 -0000 > On Oct 19, 2016, at 00:48, "damian@damianek.be" wrote: > > Hello. > > FreeBSD 10.3-STABLE amd64 1003508 > > In my /etc/src.conf i have 'WITHOUT_ICONV=yes' and i try buildworld FreeBSD > 11.0-STABLE, but: > > > ===> lib/libc/tests/iconv (all) > (cd /usr/src/lib/libc/tests/iconv && DEPENDFILE=.depend.iconvctl_test > NO_SUBDIR=1 /usr/obj/usr/src/make.amd64/bmake -f > /usr/src/lib/libc/tests/iconv/Makefile _RECURSING_PROGS=t > PROG=iconvctl_test ) > echo iconvctl_test.full: /usr/obj/usr/src/tmp/usr/lib/libc.a > /usr/obj/usr/src/tmp/usr/lib/libprivateatf-c.a >> .depend.iconvctl_test > cc -O2 -pipe -march=native -g -MD > -MF.depend.iconvctl_test.iconvctl_test.o -MTiconvctl_test.o -std=gnu99 > -fstack-protector-strong -Qunused-arguments -c > /usr/src/lib/libc/tests/iconv/iconvctl_test.c -o iconvctl_test.o > /usr/src/lib/libc/tests/iconv/iconvctl_test.c:30:10: fatal error: 'iconv.h' > file not found > #include > ^ > 1 error generated. > *** Error code 1 > > WITHOUT_ICONV option is respected and iconv is not compiled, but in "stage > 4.3: building everything" still this iconv is needed. Thank you for the report. I'll add this to my TODO list for this weekend. Cheers! -Ngie > -- > damian@damianek.be > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Wed Oct 19 19:00:02 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2249EC18C36 for ; Wed, 19 Oct 2016 19:00:02 +0000 (UTC) (envelope-from labeachgeek@gmail.com) Received: from mail-yw0-x236.google.com (mail-yw0-x236.google.com [IPv6:2607:f8b0:4002:c05::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CEA27B7F for ; Wed, 19 Oct 2016 19:00:01 +0000 (UTC) (envelope-from labeachgeek@gmail.com) Received: by mail-yw0-x236.google.com with SMTP id w3so23321004ywg.1 for ; Wed, 19 Oct 2016 12:00:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=emSCnQhcccof3U0Q2j2u5jsp/pg2BN0JvJlZFt+zl/0=; b=K1pVMbBhQyyZMNvWc0VYPi36MIs4hSAKaIqq5I3YbCk6sXZkPTl908CxPLjHk3SqDm eWGs4LreafPi/kWRZ8aLJHI+4usmzwZaQQgZBEThUQrcoDaN4mdA3eGFE8lkKqe1jyUX sJ+wY7Tv4Hp0r+/1pu13I1f1ky0P9zvRy2K0Irq5bTycuaXy7Q4MpcRUpCv4Kxy3Oxh1 ecaF5dhY0cpacT6I0pR8yDKkr5DkCAWoH4RDGk5ia/jE8b4KOwQP/Omv2UCk5tumRtxQ m3rgy0aMpaL7T8qNkVh55pcbA+JWI+jxlwEN8AD5EIykdMgJQfaToRcsfV8+fQKHzZ45 u5vA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=emSCnQhcccof3U0Q2j2u5jsp/pg2BN0JvJlZFt+zl/0=; b=GQzP6JTPQalBwpAYORF55xx57ex1SBOtR+O9XAqd3YrZfrj7u6kG7XPQbwJSBQ84LF /uv/nWUs5/prdxHnmp/AUetVPQyGWYhP3VhOtcOpPj2klkkvA2qq9YnKC+B5NdyLTgGh 5ydpedXGQZzLMb5IpeKCqWQwKI2FvBbb+Nt0Iezo2JCVZbgLPd4C+MYK7dFspg0qwCI1 8RYWY9PsSSgDjpKn4VZrqyJhaRjomqBeaxjEkc6AuP1sIOh/g1FRf0CTnjFGgh2goWVJ RT7UtbwGlCi8fL9jf+mbgIHx7YXEsCfNKGakZTwNSJmO6vPit/o+X4kWZy8K6pIgjOzt 5NRw== X-Gm-Message-State: AA6/9RlXpi8MIT4mURgzzb9IkMElUTFmaGoKZKtJ2NXTkknbuUBdQ8O2u1k7dF/LyMwO0JK3tUuGx/aymt1Djw== X-Received: by 10.202.106.65 with SMTP id f62mr5667837oic.68.1476903600817; Wed, 19 Oct 2016 12:00:00 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.244.12 with HTTP; Wed, 19 Oct 2016 12:00:00 -0700 (PDT) Received: by 10.202.244.12 with HTTP; Wed, 19 Oct 2016 12:00:00 -0700 (PDT) In-Reply-To: References: From: Beach Geek Date: Wed, 19 Oct 2016 14:00:00 -0500 Message-ID: Subject: Attacking Branch Predictors to Bypass ASLR To: freebsd-hackers@freebsd.org X-Mailman-Approved-At: Wed, 19 Oct 2016 19:11:51 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 19:00:02 -0000 This came across my tech news feed. It's a bit early and more testing is being done, but I wanted to start a discussion about it. Does this affect FreeBSD? If so, severity? Can this be countered/fixed in the OS? Link to 13 page paper: http://www.cs.ucr.edu/~nael/pubs/micro16.pdf Quotes: "Today, ASLR-based defenses are widely adopted in all major Operating Systems (OS), including Linux [17], Windows [18] and OS X [19]. Smartphone system software such as iOS [20] and Android [13] also use ASLR." "We demonstrate that our attack can reliably recover kernel ASLR in about 60 milliseconds when performed on a real Haswell processor running a recent version of Linux. Finally, we describe several possible protection mechanisms, both in software and in hardware." Opinions of whether this is a viable hack against FreeBSD systems? BG From owner-freebsd-hackers@freebsd.org Wed Oct 19 20:01:30 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3A4DFC17DDE for ; Wed, 19 Oct 2016 20:01:30 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-it0-f51.google.com (mail-it0-f51.google.com [209.85.214.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EFAA31367 for ; Wed, 19 Oct 2016 20:01:29 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-it0-f51.google.com with SMTP id 4so125393970itv.0 for ; Wed, 19 Oct 2016 13:01:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc:content-transfer-encoding; bh=+2VqKjDEEODjXK/XIf/witkvmXvJUfHjx5yJMhU8guM=; b=XA6z5CNlXLMXtQ6DsJCtuHsOSaDFRTWMnABa+BuGS0KiXFIoKMjNXYw70J6mazCBkS SCNrmnvWVZwsyqhWwZyu16HbRMYE0CVXWj2vV1Y8O2E111gNs6i+vdJhDbecp4aK7AsH Mj8i/kFQtXduHO2V/8Hdx1YusJRKoMQp30Yv3BQ6ZTpSqbQe2DzSR7vOahoAcDSyCAbc q+3USou4DGvN2PH0N3ApW+ineGYrTbcnDwoaGwh47S1xj4hPat66eRpAqtx1xmSwmucF kObiTNGVItLhKvj+txolBwj/cntaenbkjdgq3qDcVh0Ud/VSIm3SX3mmo7TJLYFeTDal VKHw== X-Gm-Message-State: AA6/9RmL/UVb2TpDGAr2mT0IHdq7QuuobPmOBIibWOj3/Xh4r5WutV46IeTGhQPeXqYtcA== X-Received: by 10.107.142.134 with SMTP id q128mr8362419iod.109.1476907281101; Wed, 19 Oct 2016 13:01:21 -0700 (PDT) Received: from mail-it0-f44.google.com (mail-it0-f44.google.com. [209.85.214.44]) by smtp.gmail.com with ESMTPSA id b6sm6757166iob.4.2016.10.19.13.01.20 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Oct 2016 13:01:20 -0700 (PDT) Received: by mail-it0-f44.google.com with SMTP id 139so117163817itm.1 for ; Wed, 19 Oct 2016 13:01:20 -0700 (PDT) X-Received: by 10.36.130.7 with SMTP id t7mr4517914itd.104.1476907278964; Wed, 19 Oct 2016 13:01:18 -0700 (PDT) MIME-Version: 1.0 Reply-To: cem@freebsd.org Received: by 10.36.220.199 with HTTP; Wed, 19 Oct 2016 13:01:18 -0700 (PDT) In-Reply-To: References: From: Conrad Meyer Date: Wed, 19 Oct 2016 13:01:18 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Attacking Branch Predictors to Bypass ASLR To: Beach Geek Cc: FreeBSD Hackers Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 20:01:30 -0000 On Wed, Oct 19, 2016 at 12:00 PM, Beach Geek wrote: > This came across my tech news feed. It's a bit early and more testing is > being done, but I wanted to start a discussion about it. > > Does this affect FreeBSD? > If so, severity? > Can this be countered/fixed in the OS? > > Link to 13 page paper: > http://www.cs.ucr.edu/~nael/pubs/micro16.pdf Hi, FreeBSD doesn't have an ASLR implementation to bypass. So the straightforward answer is no. It does not affect FreeBSD's existing code. There is an open question of whether it affects or obviates Konstantin's userspace ASLR patch which is waiting to be merged. The paper suggests a really lame and difficult software mitigation on page 10. On page 11 it suggests a possible HW mitigation, but that does not yet exist in any CPU of course. The userspace ASLR attack is somewhat limited. Key quotes: > Our prototype code tests 100 addresses in a second. 2^18 / 100 ~=3D 2^7 or ~2^11 seconds is about 35 minutes. > Please note that current BTB addressing scheme (as used in Haswell proces= sor used for our experiments) allows us to recover only a limited number of= ASLR bits. The number of bits that are randomizes is implementation specif= ic. However, according to [47], the full ASLR in Linux randomizes 12th to 4= 0th bits of the virtual address. Since 30th and higher bits are not used in= BTB addressing, only 18 bits can be recovered using the BTB attack on Hasw= ell. So it seems ASLR may still be somewhat useful on amd64, especially if bits above 30 are randomized (Haswell anyway). But it may be completely useless on i386. Best, Conrad From owner-freebsd-hackers@freebsd.org Thu Oct 20 12:26:08 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B321BC186B7 for ; Thu, 20 Oct 2016 12:26:08 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3763B95D; Thu, 20 Oct 2016 12:26:08 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u9KCQ1gT037793 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 20 Oct 2016 15:26:01 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u9KCQ1gT037793 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u9KCPx9B037776; Thu, 20 Oct 2016 15:25:59 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 20 Oct 2016 15:25:59 +0300 From: Konstantin Belousov To: Conrad Meyer Cc: Beach Geek , FreeBSD Hackers Subject: Re: Attacking Branch Predictors to Bypass ASLR Message-ID: <20161020122559.GO54029@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 12:26:08 -0000 On Wed, Oct 19, 2016 at 01:01:18PM -0700, Conrad Meyer wrote: > On Wed, Oct 19, 2016 at 12:00 PM, Beach Geek wrote: > > This came across my tech news feed. It's a bit early and more testing is > > being done, but I wanted to start a discussion about it. > > > > Does this affect FreeBSD? > > If so, severity? > > Can this be countered/fixed in the OS? > > > > Link to 13 page paper: > > http://www.cs.ucr.edu/~nael/pubs/micro16.pdf > > Hi, > > FreeBSD doesn't have an ASLR implementation to bypass. So the > straightforward answer is no. It does not affect FreeBSD's existing > code. > > There is an open question of whether it affects or obviates > Konstantin's userspace ASLR patch which is waiting to be merged. > > The paper suggests a really lame and difficult software mitigation on > page 10. On page 11 it suggests a possible HW mitigation, but that > does not yet exist in any CPU of course. > > The userspace ASLR attack is somewhat limited. Key quotes: > > > Our prototype code tests 100 addresses in a second. > > 2^18 / 100 ~= 2^7 or ~2^11 seconds is about 35 minutes. > > > Please note that current BTB addressing scheme (as used in Haswell processor used for our experiments) allows us to recover only a limited number of ASLR bits. The number of bits that are randomizes is implementation specific. However, according to [47], the full ASLR in Linux randomizes 12th to 40th bits of the virtual address. Since 30th and higher bits are not used in BTB addressing, only 18 bits can be recovered using the BTB attack on Haswell. > > So it seems ASLR may still be somewhat useful on amd64, especially if > bits above 30 are randomized (Haswell anyway). But it may be > completely useless on i386. ASLR is almost useless obfuscation scheme, which does not give much, except possibly a buzzword compliance. In fact, it does have the value, by allowing you to note persons who promote it, and to make required conclusions. It seems that finding a clever tiny trick that circumvents whatever obfuscation provided by any ASLR implementation, is the required initiation step for the low-level security researchers. Reading the interesting pdf that was linked in the initial mail, I admit that I was more impressed by the recent presentation about using PREFETCH instructions to recover kernel address map in presence of KASLR, see https://www.blackhat.com/docs/us-16/materials/us-16-Fogh-Using-Undocumented-CPU-Behaviour-To-See-Into-Kernel-Mode-And-Break-KASLR-In-The-Process.pdf This is mostly equivalent thing, using cache timing instead of BTB timing. You will quickly find a dozen or two of the similar 'exploits' by Googling for several minutes. From owner-freebsd-hackers@freebsd.org Thu Oct 20 20:58:42 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6A319C1BD7F for ; Thu, 20 Oct 2016 20:58:42 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-yb0-x233.google.com (mail-yb0-x233.google.com [IPv6:2607:f8b0:4002:c09::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2F8C3C90 for ; Thu, 20 Oct 2016 20:58:42 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: by mail-yb0-x233.google.com with SMTP id 184so33291960yby.2 for ; Thu, 20 Oct 2016 13:58:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=Ei+xQfdWzvUXLEFpPM+x1cdQUnHLo62e4i9Q6xghoUc=; b=KTHQYKH0stqdqNlBvUqRDhrHx3SXKFIjWShk7oRD3ozr+JnQvAJPfa2X0kHW0g7UJ3 bdnGZ/JeX3BDaK4+rLakrKiTKkZNzx4u15ec4pN+LSKqK7ifFhM6DI63vcDnr2yk1DMk OnP4VIUdgvZsY5TBBJP0wMQugLrJogjgsPzVVpRG8brLGP67hiY+FGOt97ihi/PqYfbJ fH/TXleWR5GnRmyOzOs6ar7677u8+h1fMpi1qDWvr9ZddSb+wj4EWpH9tuLe9f6GeqLu 3+dxKkdK/9CjDn5xQfs6jpvKkcUhavdv9keJ4jlHlrPTNlk86qIt/qs1l4ydlco60UVh Oqpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Ei+xQfdWzvUXLEFpPM+x1cdQUnHLo62e4i9Q6xghoUc=; b=O5kffZGTgFHWtfAMOrdLIBXwu8NX0sCVoV7ZnQknrlXsGw3aPbEjXjrJLXGc6632xE KCuA7cSo+TGEM1CcXOyW+DnIYaR0TodqHkCAFPFkDDahx/0moSjrJd7yv1DCNpg7JWYM AsSB0HwWCWHnWDUht0nr1KP2fsQCxtoQ0nF3VzfMcVmJeNES1A7dPXM569MOCdTtKC6R s7wJxXAKXFUckkayXjVFpx+BKabAl+XjiWCtKPhlRavLCDR0LX34RGEy6fCQYCPVfUaT qqpsZ8VjWX9XInjJJ4xEAXuTx9gJZEEVFybByQDDrVb4JKd7/6hTQMLlLu69qO+5jKD2 ttaw== X-Gm-Message-State: AA6/9RlATNRdPGZZS00p+qZW1lUbjuTDRS2X5Nv4UPP+EPcoPzWupOE3d86I8QLnJDKEjjWUzQZ3ipeSXK3QyQ== X-Received: by 10.37.5.19 with SMTP id 19mr4354748ybf.22.1476997121195; Thu, 20 Oct 2016 13:58:41 -0700 (PDT) MIME-Version: 1.0 Received: by 10.37.161.37 with HTTP; Thu, 20 Oct 2016 13:58:40 -0700 (PDT) From: Zaphod Beeblebrox Date: Thu, 20 Oct 2016 16:58:40 -0400 Message-ID: Subject: livelock without explaination. To: FreeBSD Hackers Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 20:58:42 -0000 I have a FreeBSD 10.3 server that houses a 16 disk ZFS array, runs samba, and runs a large torrent client (a few hundred torrents, but probably less than 50 of them active). Every so often it goes into a livelock. The pipe from the location is 50/10 ... and the torrents are limited to 15/5. Inside the building, GigE access through SMB or NFS isn't uncommon, but not terribly constant. Watching movies happens over SMB, but I hardly think that's a stretch. ... but every 3 to 5 days, I get livelock. Even the console locks, although ... often... after some minutes, I can get a CTRL-ALT-DEL to register and get it shut down properly. It's also true that stopping rtorrent will unfreeze the livelock, but it will return quickly if restarted. I've tried reducing the number of active torrents. I've also looked at the netstat -m 430/15965/16395 mbufs in use (current/cache/total) 21/2065/2086/2034610 mbuf clusters in use (current/cache/total/max) 21/2003 mbuf+clusters out of packet secondary zone in use (current/cache) 128/249/377/1017304 4k (page size) jumbo clusters in use (current/cache/total/max) 256/2775/3031/301423 9k jumbo clusters in use (current/cache/total/max) 0/0/0/169550 16k jumbo clusters in use (current/cache/total/max) 2965K/34092K/37057K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile ... now I'd like to increase mbufs there ... but: [1:5:305]root@vr:~> sysctl -a | grep nmb kern.ipc.nmbufs: 13021500 kern.ipc.nmbjumbo16: 678200 kern.ipc.nmbjumbo9: 904269 kern.ipc.nmbjumbop: 1017304 kern.ipc.nmbclusters: 2034610 ... so... odd... and why would this all effect typing on the keyboard ... unless there's some interaction between the disk controllers and this lot? The root is UFS on a SATA gmirrored with a PATA drive (legacy). This that really bad (tm)? From owner-freebsd-hackers@freebsd.org Fri Oct 21 04:55:06 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F1068C1B192 for ; Fri, 21 Oct 2016 04:55:06 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pf0-x230.google.com (mail-pf0-x230.google.com [IPv6:2607:f8b0:400e:c00::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BD60CCB4 for ; Fri, 21 Oct 2016 04:55:06 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mail-pf0-x230.google.com with SMTP id r16so49696538pfg.1 for ; Thu, 20 Oct 2016 21:55:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:from:in-reply-to:date:cc:message-id:references :to; bh=hLbd6hsWIvLfaDhQQWK8GqiB8HpmcgTJEf8ZURjzEvI=; b=lZlib5VpU2fd/QfpRfmgClW8tiGegF3JZKpE1sA29BcMbPKt8ajfDq/XmamLX8iwYO L/wAiIcgaR05E0nhTPzAVAMR3n9dVlnyt6JsocvR8fFMil4MrtTxHkXjnqwXIthZVODU KsAgxfWIEQzAkxWcX5LbDcIQjmu8ftJ1e4XM+ZgtqkGRYsAH+D3ZyVcg0wJ0rM/EjY6C FHDSydb0kNUNWfqXg7jdyrf+fgDiqsB3J5RJfUbjYSrEt3RIddN3D4RHStf3+mEy2plc H/ATA6hWI9Nle3yjFF+ApNaglmIYjXHsoHTMTIjlF19MJ8Z7X9Hwqb5wZ4yUbUXb1EN3 50TA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:mime-version:from:in-reply-to:date:cc :message-id:references:to; bh=hLbd6hsWIvLfaDhQQWK8GqiB8HpmcgTJEf8ZURjzEvI=; b=UArvA42/0f2PNOaW3atIfF62XHS2hW/SOOpmDO9W5vavVCjw/dEVO6Cmd3jOoep/jA CGKu47l4kg+hnQgqtDshUykvpXQGkE1WiZF22Vrg22HikP0vcaS9ljo2RAF9Hq9RDZJl 8AICETjVoLcIOPiaCe/xJ2LT8nYcwjrYwAwB8CRMDBGcXY2A3r6aAFL/aWfs41codYNF AwbDc9PDUM/k4YIEch/ZVkQx9zSKokmLipblzQfBDx9G5sQAD1OWGAkkzZDL3QbbH07q EYia2dWTQJtQdllTK14boK04OTpx9JUU7EqOw5GcA85Li1Y6mBiXvWmtjbovLQ5w3Pc7 NJrQ== X-Gm-Message-State: AA6/9RlgjrZsPhVV6LvboHxqVQgc4RaDcJB3t7ErBSxeB2rcxa8kp7saHSAYJpXKKZdkvA== X-Received: by 10.99.94.196 with SMTP id s187mr6396080pgb.107.1477025706310; Thu, 20 Oct 2016 21:55:06 -0700 (PDT) Received: from [192.168.20.12] (c-73-19-52-228.hsd1.wa.comcast.net. [73.19.52.228]) by smtp.gmail.com with ESMTPSA id 4sm973198pfg.66.2016.10.20.21.55.05 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 20 Oct 2016 21:55:05 -0700 (PDT) Subject: Re: FreeBSD11 buildworld WITHOUT_ICONV Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: multipart/signed; boundary="Apple-Mail=_84CB370F-8400-4588-A2AC-70090F571425"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail From: "Ngie Cooper (yaneurabeya)" In-Reply-To: Date: Thu, 20 Oct 2016 21:55:08 -0700 Cc: freebsd-hackers@freebsd.org Message-Id: <75FC0C9B-F70B-4524-856F-9C8F4EB45776@gmail.com> References: <30603328-0967-4F43-AF8F-1291439DEEE3@gmail.com> To: damian@damianek.be X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 04:55:07 -0000 --Apple-Mail=_84CB370F-8400-4588-A2AC-70090F571425 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Oct 20, 2016, at 03:30, damian@damianek.be wrote: >=20 > 2016-10-19 19:51 GMT+02:00 Ngie Cooper : > > WITHOUT_ICONV option is respected and iconv is not compiled, but in = "stage > > 4.3: building everything" still this iconv is needed. >=20 > Thank you for the report. I'll add this to my TODO list for this = weekend. > Cheers! >=20 > Great, thanks. Just fixed via r307700 =E2=80=94 thank you! -Ngie --Apple-Mail=_84CB370F-8400-4588-A2AC-70090F571425 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJYCZ+sAAoJEPWDqSZpMIYVhQoQALg2t5gYhBFCf6SMTWaiPG9J /M2nhONdXmrrAgMofw9eTqMqjLlcgj3wQkz8D/G3Aqkbr5QJ5Z2iFfO3lO7iKd+P pQMY8oVSFRClP04LuEDgGRxgdgIzdik3cSmIr2aY1uB+B3eVKKLpoyma9qRBc3+b MQeZvUzqgr+02KtRcixYMpswJBZiYqUwhRMzNnXzNvsOU/seEZNm7e832rOUjMgJ iiXlgqSTPQ9h4vwW8j2Z10f8ATMPHcrFhMNVDHvhNV+i4nBfmeo2GbiNkpgHDx2l 5X0cty8OIva8PJ7C58utEkyvs/8KxOpnDOL7J4hSCPoZ6zt0FLWnO3+npMLaa/Nw YYBjhYcezGxNhl3DlRxXt7M58R10MT7kQ0IJtXCSKnfWHUkjpSeTNqY4db2nDSQt YB5kYWwIMml5zq576WyJ6LUwSPXlOpDvIkLKLJtUaiM4/GcyDLFQo5oX22gaTx7r 6eH5bPKTtV3hTvSe4d/fr0huz73duLSrMl+4DZhWKzrTLbQI3cCIanVapNaIwZ9I /Kkl1Ke9elurfGwqD8ImwTLGstP6gFOXbPz6ULNffICFLrieksc7BHZ+m2katZB5 Yasinr5z4+YiqLAj92xAd6XoBPJ6ExdfbZKbBLirZ+R1d/FR7m1ejEnjHqNGa6qh lC4y37jWR2mq9ApU5h/S =pOIb -----END PGP SIGNATURE----- --Apple-Mail=_84CB370F-8400-4588-A2AC-70090F571425-- From owner-freebsd-hackers@freebsd.org Thu Oct 20 10:31:09 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CF959C18800 for ; Thu, 20 Oct 2016 10:31:09 +0000 (UTC) (envelope-from damian@damianek.be) Received: from mail-it0-x232.google.com (mail-it0-x232.google.com [IPv6:2607:f8b0:4001:c0b::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9FD06CFF for ; Thu, 20 Oct 2016 10:31:09 +0000 (UTC) (envelope-from damian@damianek.be) Received: by mail-it0-x232.google.com with SMTP id 66so82680597itl.1 for ; Thu, 20 Oct 2016 03:31:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=damianek-be.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=IiP3nziuI4Zo5ZoboSThbUACKQWEMq6gh32yzSTJfQ4=; b=v3wLdowTcG2wJC7Ln0femdQEjafIHwjTF9bqSku5R6vBCz3AM85khUT/bqw/5VYtaA eii2FkI5D305PUzCkvi58+agPhxYvlETTJ86WJugOD7hUdJvSBx+voS8xJKcn0l5l5kw buQUt3K+CuK+vqbCdTgoHe5l/HQw+04NjUuJKliMTkRbhcLurOE/G6QHrM81gPEpU9/Y Lsy0HDRlKWJsz2/FPhk6JHMrHQyeWBH/Gs9dM/mA2rY64F3/JUqhqMuOSN/f3oOZ3oC9 7JhaJG2g7820/OSNg1FZ6uOPetzdtMQbsKF3nasXktS4Tou2Sb8o+lg6tXOR38u+YwC2 E6Ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=IiP3nziuI4Zo5ZoboSThbUACKQWEMq6gh32yzSTJfQ4=; b=aZVtAUCBruua5fjkmNlpJBCTSx8U2U7OB745h2dEUUC3NunxyU8zv+gSS+hDG3kYn7 jdI8MIC9qYWgZd11a3TRKAD1l8+QYudf99ZW68vjr/EVc1du3hhb8iR2FkTloOYhAzJC mmjVCi5Z/GmCq7WZCL4ba9MvBU9g2epKf1IxG5iLlHfcGMXo7UQVW0lDgGzejY/aBMj9 cU7LmUKoTvkaynK850tY7T33oEGYNwBdRy8ho2KOC8v0XImPSC5/BtolJT4JZ7SaogtN B9Qm3xlzRXViuc43zW4ZywUhSNQlQpeX6HWQswfmLjKBo+IXHvVURDK8XmkKV0Ta8s0h miFg== X-Gm-Message-State: AA6/9RnTWhKoVmR/CGIXSgV9I6ykfgye9bFj7TjN6fra0X8lj9CrJe0mF6H3b4nLEZO7wRI4CBtxWRpy1OQ3WQ== X-Received: by 10.107.1.136 with SMTP id 130mr11144600iob.72.1476959468361; Thu, 20 Oct 2016 03:31:08 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.39.71 with HTTP; Thu, 20 Oct 2016 03:30:28 -0700 (PDT) In-Reply-To: <30603328-0967-4F43-AF8F-1291439DEEE3@gmail.com> References: <30603328-0967-4F43-AF8F-1291439DEEE3@gmail.com> From: "damian@damianek.be" Date: Thu, 20 Oct 2016 12:30:28 +0200 Message-ID: Subject: Re: FreeBSD11 buildworld WITHOUT_ICONV To: Ngie Cooper Cc: freebsd-hackers@freebsd.org X-Mailman-Approved-At: Fri, 21 Oct 2016 11:12:27 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 10:31:09 -0000 2016-10-19 19:51 GMT+02:00 Ngie Cooper : > > WITHOUT_ICONV option is respected and iconv is not compiled, but in > "stage > > 4.3: building everything" still this iconv is needed. > > Thank you for the report. I'll add this to my TODO list for this weekend. > Cheers! > Great, thanks. -- damian@damianek.be From owner-freebsd-hackers@freebsd.org Fri Oct 21 13:20:11 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DBACEC1B4DE for ; Fri, 21 Oct 2016 13:20:11 +0000 (UTC) (envelope-from damian@damianek.be) Received: from mail-it0-x233.google.com (mail-it0-x233.google.com [IPv6:2607:f8b0:4001:c0b::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ACB3C769 for ; Fri, 21 Oct 2016 13:20:11 +0000 (UTC) (envelope-from damian@damianek.be) Received: by mail-it0-x233.google.com with SMTP id m138so164484717itm.0 for ; Fri, 21 Oct 2016 06:20:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=damianek-be.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=IZbMpnYB9jTa80cha8I5TjDM8A9skzvHvJ+Bm18ACAM=; b=kdLACJpeZbC6lWZaK52R8HOwaP7aoEHd/emmkLLBSy7tb+iPF4DWGsvceDyhmqDXsc ZSE31ITa5pxc645/59EbDh6wzNPYYAhdyGAub8103UsYMLJG8RpysYKoES37fYOGn9nk a2/oWMuhjn3UsCZWbnLaigQum2Fhz8NPyiaQW2ZphTao07seInlYcO4Ue6UpGrVbUXtt D4g487mKitdaJySSYxxvLCnB1+orqKRxT+wFgWklI5zahSHKUcAwfsM8pHSfYBUqbk9I +IDs+Gu1HINtk1qtD0hwFVNrEyREutQ/LwFJoaG5F3o1o1VkHqH62UYI0VaMKUQQAYzb 8LZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=IZbMpnYB9jTa80cha8I5TjDM8A9skzvHvJ+Bm18ACAM=; b=RqVqrEwErAds0Fghq0dbgG1cN2zAZNwcin6P9LtD2hdIit1OsacV9Xh99KoaONdgvE E+/xQyQUurEVwmHsLkUTQkhIlQL4/KRkaXbVpoHTcuwD521n8i5rcVpjtvP3eU6qbfce TLHLz17Lqo2C4/r0WRjp1aXuT+xfpFBmURq2jgogUycFKR739vP4zjuy+a7JqDQuHd03 bEBtZBV2kFnPgcgSXJk/x/JGz2x9jQhRrnImgkeJJ52tD6/6BzxEIXOTkaDiWFutMfyD UwWRNYkMg/0QLXpSyZnSZTTg91IprMB9bjsNrN4GFb9gXpSNXBXJgXZHhH3bELy7cCB7 ICpA== X-Gm-Message-State: AA6/9RkIF3u61glWTlZGMQ2zMMT4OlsDtK8q2F4zm0F94yYsO0nHdu12rSpyZy7cVJqXjfaiQ7RqDqSToIUshw== X-Received: by 10.36.249.131 with SMTP id l125mr14091840ith.113.1477056010868; Fri, 21 Oct 2016 06:20:10 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.39.71 with HTTP; Fri, 21 Oct 2016 06:19:30 -0700 (PDT) In-Reply-To: <75FC0C9B-F70B-4524-856F-9C8F4EB45776@gmail.com> References: <30603328-0967-4F43-AF8F-1291439DEEE3@gmail.com> <75FC0C9B-F70B-4524-856F-9C8F4EB45776@gmail.com> From: "damian@damianek.be" Date: Fri, 21 Oct 2016 15:19:30 +0200 Message-ID: Subject: Re: FreeBSD11 buildworld WITHOUT_ICONV To: "Ngie Cooper (yaneurabeya)" Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 13:20:11 -0000 2016-10-21 6:55 GMT+02:00 Ngie Cooper (yaneurabeya) = : > > Just fixed via r307700 =E2=80=94 thank you! > Rev 307725 Still same problem: /usr/src/lib/libc/tests/iconv/iconvctl_test.c:30:10: fatal error: 'iconv.h' file not found --=20 damian@damianek.be From owner-freebsd-hackers@freebsd.org Fri Oct 21 15:34:20 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C8659C1B77F for ; Fri, 21 Oct 2016 15:34:20 +0000 (UTC) (envelope-from labeachgeek@gmail.com) Received: from mail-yw0-x231.google.com (mail-yw0-x231.google.com [IPv6:2607:f8b0:4002:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 80960ED; Fri, 21 Oct 2016 15:34:20 +0000 (UTC) (envelope-from labeachgeek@gmail.com) Received: by mail-yw0-x231.google.com with SMTP id t193so100061181ywc.2; Fri, 21 Oct 2016 08:34:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=8l68HrywYe4MwkFnS8dUiAQnBpD/rUQglK/AzLXVQlI=; b=wwxogk6viyLcpWEXr/nuaQnsV4PLXp9UPMCne8AnKAjX5F7um94sE1t/drpJ4I9/Kh rniTpzPdBs9y3Uq8EpfuXyT7cI3QPE1sOJzp+YPqNY5LlbHo4xNDkQQukDSwWldKx5RF dL38TscRjmbYxQCMpTaPJaFgogWgNlWSslgOrTPZebmxtPTfrMujFu+3qyLfJ20/Ys2M 93vw4BO34oY/xzRsACKiBi+YiY2xTyaCUHN9k7Nr6p2+rlWzQ4wmAKqjVnBGqX5hP9gm i/1uHDEA+bT64Zgv9//M4PGqfGIr0nala0SJnbaeiAsmZT8yctxmq66aWKnXnKKlWkOn fVRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=8l68HrywYe4MwkFnS8dUiAQnBpD/rUQglK/AzLXVQlI=; b=PRs1c326yHCEERTIU+08fSCoRbsJrIw2ptN6kYr0hzMkP6zik5LYRtNcqw3s/jStEy HOxTF/J7KpqiGDiYjsDaJLzZhwUXEqIUbLtUMzRwU1NXhNTYaL55ccpo+ufQ7tD7LD++ 2s1EEb00OcCbN+hkXOLyRUBPSFSf0W6MoxZ90IVpW4uipHAnVcqrNe1CsgvwtcwAggwg +gPb0uCmukFGiTFWBOyDbZSBbcQjkJla7ZkGvSIWQSyEF3vJZmCQ0YMmktvSXqmkjJEh nUyCh8U1gFEoCxvhUEHwNyqWqtutICl3Rd6lq8SJ6ZTNmEYLLZqH93EvhTaAfmxK6cuA Hzmg== X-Gm-Message-State: AA6/9RnBLPDu5p+MYAURDik868I/Qi4UAtQTw0KXpxoaiLAMkW+yyj7THWbbiP+/QY976oTdpE9NuwxY5Dqbbg== X-Received: by 10.202.196.204 with SMTP id u195mr11846286oif.150.1477064059626; Fri, 21 Oct 2016 08:34:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.220.85 with HTTP; Fri, 21 Oct 2016 08:34:18 -0700 (PDT) Received: by 10.202.220.85 with HTTP; Fri, 21 Oct 2016 08:34:18 -0700 (PDT) In-Reply-To: <20161020122559.GO54029@kib.kiev.ua> References: <20161020122559.GO54029@kib.kiev.ua> From: Beach Geek Date: Fri, 21 Oct 2016 10:34:18 -0500 Message-ID: Subject: Re: Attacking Branch Predictors to Bypass ASLR To: Konstantin Belousov Cc: freebsd-hackers@freebsd.org, Conrad Meyer X-Mailman-Approved-At: Fri, 21 Oct 2016 16:30:03 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 15:34:20 -0000 On Oct 20, 2016 07:26, "Konstantin Belousov" wrote: > > On Wed, Oct 19, 2016 at 01:01:18PM -0700, Conrad Meyer wrote: > > On Wed, Oct 19, 2016 at 12:00 PM, Beach Geek wrote: > > > This came across my tech news feed. It's a bit early and more testing is > > > being done, but I wanted to start a discussion about it. > > > > > > Does this affect FreeBSD? > > > If so, severity? > > > Can this be countered/fixed in the OS? > > > > > > Link to 13 page paper: > > > http://www.cs.ucr.edu/~nael/pubs/micro16.pdf > > > > Hi, > > > > FreeBSD doesn't have an ASLR implementation to bypass. So the > > straightforward answer is no. It does not affect FreeBSD's existing > > code. > > > > There is an open question of whether it affects or obviates > > Konstantin's userspace ASLR patch which is waiting to be merged. > > > > The paper suggests a really lame and difficult software mitigation on > > page 10. On page 11 it suggests a possible HW mitigation, but that > > does not yet exist in any CPU of course. > > > > The userspace ASLR attack is somewhat limited. Key quotes: > > > > > Our prototype code tests 100 addresses in a second. > > > > 2^18 / 100 ~= 2^7 or ~2^11 seconds is about 35 minutes. > > > > > Please note that current BTB addressing scheme (as used in Haswell processor used for our experiments) allows us to recover only a limited number of ASLR bits. The number of bits that are randomizes is implementation specific. However, according to [47], the full ASLR in Linux randomizes 12th to 40th bits of the virtual address. Since 30th and higher bits are not used in BTB addressing, only 18 bits can be recovered using the BTB attack on Haswell. > > > > So it seems ASLR may still be somewhat useful on amd64, especially if > > bits above 30 are randomized (Haswell anyway). But it may be > > completely useless on i386. > > ASLR is almost useless obfuscation scheme, which does not give much, > except possibly a buzzword compliance. In fact, it does have the value, > by allowing you to note persons who promote it, and to make required > conclusions. > > It seems that finding a clever tiny trick that circumvents whatever > obfuscation provided by any ASLR implementation, is the required > initiation step for the low-level security researchers. Reading the > interesting pdf that was linked in the initial mail, I admit that > I was more impressed by the recent presentation about using PREFETCH > instructions to recover kernel address map in presence of KASLR, see > https://www.blackhat.com/docs/us-16/materials/us-16-Fogh-Using-Undocumented-CPU-Behaviour-To-See-Into-Kernel-Mode-And-Break-KASLR-In-The-Process.pdf > This is mostly equivalent thing, using cache timing instead of BTB > timing. You will quickly find a dozen or two of the similar 'exploits' by > Googling for several minutes. Thank you for the responses. I was sure I'd get questions about this, and they just started coming in today. We're in the process of hardware upgrades and handling that has all my attention and time currently. Thanks again, BG From owner-freebsd-hackers@freebsd.org Fri Oct 21 21:12:52 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 109B1C1C8A2 for ; Fri, 21 Oct 2016 21:12:52 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pf0-x22b.google.com (mail-pf0-x22b.google.com [IPv6:2607:f8b0:400e:c00::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C4F35CBE for ; Fri, 21 Oct 2016 21:12:51 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mail-pf0-x22b.google.com with SMTP id 128so63245369pfz.0 for ; Fri, 21 Oct 2016 14:12:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:from:in-reply-to:date:cc:message-id:references :to; bh=2/JZbrZh2/lut/7ObCLsirvi/5TCviOTOuJtvT2nqjY=; b=YiT6Wp2hor+uQZXmjAR9OKeqMusCL66ZAxtma8zJ8LlDFbF5kXSotvwsnmb3HFWBSv OKklevNB8Sga+RkVSNAAr3eu25hgzyIyf/Ej9zqLDYFjEl6SFwUz6eBvWbdYLqidOPSA HdCOiVIk/T+qTrAln1ACry1VASiov370SvYY0jrerRnm47ylLX14kzuq8bW6KEK6l56y BG8467xDx+mDk3AX5F4mAI9Hjk2JZ7nafLZoZwD3SibLSnAyJK6WD/hINwqr+KPHC70Y S9MwisadwsSaJPSn2x7zKqFrtOd6oTuREKFAsg1EJ5Ld69YAB8iqtLhvJseksHcHabma cffQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:mime-version:from:in-reply-to:date:cc :message-id:references:to; bh=2/JZbrZh2/lut/7ObCLsirvi/5TCviOTOuJtvT2nqjY=; b=GBxMDZJNPyxi4VosAQzj4/3rvkKvrCKlDMSMmNSX2S3CKxh7akWPgtwhr9/17Sgq9n T8VG7OGfLsWBVefVt4wwz90YGnLo1rmYqcKPNEKRR8/sB0wRPuO9LAgckDpkoZLr+oKz uFMLeYidawcs2/CDjkLSge+lTzoPZqnFMU4Qp2f9zrn4BbTf9zo8qJ6jks6pNwK7chPL tpdu+pD/tUUL9pVGBXn/EfL/v2bEbn/CRVutpbN0+bVIO4/WYxEyEk9QNg1JAGKfOKJi RRT/Xi2x0XkozvuU+zRQB3X7J5G8mRd/Tns+GDMxfug4MaCE+nOXv6NFXrsVL+kbW8jS JzfQ== X-Gm-Message-State: ABUngvdPoa6XBaR00Y6xgaUI/EfeRN3vo3QSfdLpTuYYDtvhXULIICaqmPFvDpsvXVfpxg== X-Received: by 10.98.32.82 with SMTP id g79mr5070026pfg.142.1477084371403; Fri, 21 Oct 2016 14:12:51 -0700 (PDT) Received: from [192.168.20.12] (c-73-19-52-228.hsd1.wa.comcast.net. [73.19.52.228]) by smtp.gmail.com with ESMTPSA id yx8sm7255187pac.29.2016.10.21.14.12.50 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 21 Oct 2016 14:12:50 -0700 (PDT) Subject: Re: FreeBSD11 buildworld WITHOUT_ICONV Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: multipart/signed; boundary="Apple-Mail=_91AC85C2-1D98-4790-8BFB-9A76FAD08CF5"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail From: "Ngie Cooper (yaneurabeya)" In-Reply-To: Date: Fri, 21 Oct 2016 14:12:49 -0700 Cc: freebsd-hackers@freebsd.org Message-Id: <99EA092D-EB84-4D54-9583-F7E328466376@gmail.com> References: <30603328-0967-4F43-AF8F-1291439DEEE3@gmail.com> <75FC0C9B-F70B-4524-856F-9C8F4EB45776@gmail.com> To: "damian@damianek.be" X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 21:12:52 -0000 --Apple-Mail=_91AC85C2-1D98-4790-8BFB-9A76FAD08CF5 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Oct 21, 2016, at 06:19, damian@damianek.be wrote: >=20 > 2016-10-21 6:55 GMT+02:00 Ngie Cooper (yaneurabeya) = : >=20 > Just fixed via r307700 =E2=80=94 thank you! >=20 > Rev 307725 > Still same problem: > /usr/src/lib/libc/tests/iconv/iconvctl_test.c:30:10: fatal error: = 'iconv.h' file not found Are you trying to build from lib/libc/tests or = lib/libc/tests/iconv ? If the latter, then yes=E2=80=A6 that will not = work. The former case should work though (and I tested this out before I = committed the change by setting MK_ICONV=3Dno). Thanks! -Ngie --Apple-Mail=_91AC85C2-1D98-4790-8BFB-9A76FAD08CF5 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJYCoTRAAoJEPWDqSZpMIYVwlsQAIIOnAF79smYB0se6PCRBKHo kX1nEepb8fji3t9YVSkiqnVI1tRsIxRtJ/L9D4idxpcQJtL4b/mW9bbrCawG3FiJ K0m/YT0wYrafNj16PXvZsA+fYHLtQOf2xMw3rGpPVIi+dSgdbHHlAAmrKn5wLcQ9 9sJrVWcaORlDFDiZmfkY3YghKZ3qJFoMD8usTzMQyWs+uB5fXb4DWrza/zCDx3ob C310Z7paMsTlj6q+FhEuLb+pTdg3a680Pu29EhryBcOpWWzqBcDuXi67M2kBu+u1 R3mvJ9vXG+YwR7g4EpaU1lxYXkC2tA0CzI/QXCIBVyMcLqHJpQc0E/ZS2V1gPoQw Uc0x8JR3Qyh5evz9PvXrSNkryNEnknGF6FUc0Ji60uc98YJPV+Dqb9kLXlaVM5O7 kwRdt1FROlcuobzwBG0rwb9lIN0xkVdpc1Fvksr0sdTR4piUDyb0ZIQtl53IZtCl QFIjrk6b5Ug3us0AHz0Ppi/4vlwFUwrU+ry15DePmPIU2Qi7HceJDoWX5TJaXGx2 inStGP1oPXELz6ZnXribzbtwSBLGnyPwrcyENMwSQbDm/EuaDy5c5cZ9NusTIt1r I0Kq7s6CTEgQNLjBrSg5mUlc/EXl8pG706aH8QmWdR5RwZK9yetbyC+EU2swakgg jT6Z+dRnrpA7SlTzb44f =iSXl -----END PGP SIGNATURE----- --Apple-Mail=_91AC85C2-1D98-4790-8BFB-9A76FAD08CF5--