From owner-freebsd-arch@freebsd.org Sun Jan 31 23:02:18 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7223CA7506A for ; Sun, 31 Jan 2016 23:02:18 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 621C0896 for ; Sun, 31 Jan 2016 23:02:18 +0000 (UTC) (envelope-from jilles@stack.nl) Received: by mailman.ysv.freebsd.org (Postfix) id 5F2A8A75069; Sun, 31 Jan 2016 23:02:18 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5EC20A75068 for ; Sun, 31 Jan 2016 23:02:18 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mailhost.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2CEE1895; Sun, 31 Jan 2016 23:02:18 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from toad2.stack.nl (toad2.stack.nl [IPv6:2001:610:1108:5010::161]) by mx1.stack.nl (Postfix) with ESMTP id E515C359306; Mon, 1 Feb 2016 00:02:14 +0100 (CET) Received: by toad2.stack.nl (Postfix, from userid 1677) id C3A79892A8; Mon, 1 Feb 2016 00:02:14 +0100 (CET) Date: Mon, 1 Feb 2016 00:02:14 +0100 From: Jilles Tjoelker To: John Baldwin Cc: arch@freebsd.org Subject: Re: Refactoring asynchronous I/O Message-ID: <20160131230214.GA37435@stack.nl> References: <2793494.0Z1kBV82mT@ralph.baldwin.cx> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2793494.0Z1kBV82mT@ralph.baldwin.cx> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jan 2016 23:02:18 -0000 On Tue, Jan 26, 2016 at 05:39:03PM -0800, John Baldwin wrote: > Note that binding the AIO support to a new fileop does mean that the AIO code > now becomes mandatory (rather than optional). We could perhaps make the > system calls continue to be optional if people really need that, but the guts > of the code will now need to always be in the kernel. Enabling this by default is OK with me as long as the easy ways to get a stuck process are at least disabled by default. Currently, a process gets stuck forever if it has an AIO request from or to a pipe that will never complete. An AIO daemon should not be allowed to perform an unbounded sleep such as for a pipe (NFS server should be OK). -- Jilles Tjoelker From owner-freebsd-arch@freebsd.org Mon Feb 1 19:57:45 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E4B8FA9801A for ; Mon, 1 Feb 2016 19:57:44 +0000 (UTC) (envelope-from cturt@hardenedbsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id C8E96B03 for ; Mon, 1 Feb 2016 19:57:44 +0000 (UTC) (envelope-from cturt@hardenedbsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id C5D48A98017; Mon, 1 Feb 2016 19:57:44 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AD169A98016 for ; Mon, 1 Feb 2016 19:57:44 +0000 (UTC) (envelope-from cturt@hardenedbsd.org) Received: from mail-wm0-x22e.google.com (mail-wm0-x22e.google.com [IPv6:2a00:1450:400c:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BF3BB02 for ; Mon, 1 Feb 2016 19:57:44 +0000 (UTC) (envelope-from cturt@hardenedbsd.org) Received: by mail-wm0-x22e.google.com with SMTP id l66so86733783wml.0 for ; Mon, 01 Feb 2016 11:57:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd-org.20150623.gappssmtp.com; s=20150623; h=mime-version:date:message-id:subject:from:to:content-type; bh=oWKOTh4/JXgVwQdoHyQhbI6mbYmxrYpXdnT/HTNMR/U=; b=EJtewfZFnwENBGLJAR+IDt8BN5W1atNMT+4Xh+6OJshi8BcwIpdSobbzc2IFBBpIfa GwqAZ44cjthxAXR/wRn5fPtfrPbLjFiADnKDg/2wUkCnzQPCpH7QoOgEJfqzRYgmzp0E IU9lEUP6EhhhfUtxz1HiwYjUV51eKOUiEAIynY9OAQrFfavPsk7aZ56b/Xrb9GVYveZk rmWV0VXr1zZ2eFPqXI7oFGBhAhFG8lKnMmShlV7z9PEIz6Lc3A7rs9w4jMgAv9Rtb6TP F+LxxylF3lvHwFlC1AlKA/jgqk8vxl4immEqI4yV3oOkkmD+1wv5CJ6mfttw98gcwniz JFsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=oWKOTh4/JXgVwQdoHyQhbI6mbYmxrYpXdnT/HTNMR/U=; b=hnnsL127bHqxk/pB4I7Qq/c96x3eZr5f8jiEvWcKAiFBGIXxlJcgxIdD8GZdp6ecs9 UPh9ie8agT/z7wNgzMmPh/6mzzCFPLXrTsOPHuQKXViHWTFm+UhWea2sHGQNjSKhSyXP 2lwF4Ix34O9AsB+iiN5iPW1Cpd8nqLISI/gaCV1vLgshzA6EV1e5BQJQYdlJ6uvtrNXe mf6yxY+Hq+ze8vf1GzmKCOj1f7LVNG6YnYHTnrIpHQBtdZQro6MRDq/M13uKKzby8QnK QXSoSFHfATfloTmIsL1P8SA0LdlIEPrFmjOKPqqC5u7fgHbuX9jBZnkK+eFPD0HgnoKX tRKw== X-Gm-Message-State: AG10YORyV/FKgyGGYaBqskIfxVxPbdRYFbAI6mpr8Chp59AeelgfjZmLcmwTzVYUmiXsQP1ESn6ZQvPT/XzhHqe4 MIME-Version: 1.0 X-Received: by 10.194.21.101 with SMTP id u5mr29009098wje.53.1454356662819; Mon, 01 Feb 2016 11:57:42 -0800 (PST) Received: by 10.27.157.18 with HTTP; Mon, 1 Feb 2016 11:57:42 -0800 (PST) Date: Mon, 1 Feb 2016 19:57:42 +0000 Message-ID: Subject: OpenBSD mallocarray From: C Turt To: arch@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 19:57:45 -0000 I've recently started browsing the OpenBSD kernel source code, and have found the mallocarray function positively wonderful. I would like to discuss the possibility of getting this into FreeBSD kernel. For example, many parts of kernel code in FreeBSD use something like malloc(xxx * sizeof(struct xxx)). If xxx is 64bit and controllable by user, this allocation can easily overflow, resulting in a heap overflow later on. The mallocarray is a wrapper for malloc which can be used in this situations to detect an integer overflow before allocating: /* * Copyright (c) 2008 Otto Moerbeek * * Permission to use, copy, modify, and distribute this software for any * purpose with or without fee is hereby granted, provided that the above * copyright notice and this permission notice appear in all copies. * * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ /* * This is sqrt(SIZE_MAX+1), as s1*s2 <= SIZE_MAX * if both s1 < MUL_NO_OVERFLOW and s2 < MUL_NO_OVERFLOW */ #define MUL_NO_OVERFLOW (1UL << (sizeof(size_t) * 4)) void * mallocarray(size_t nmemb, size_t size, int type, int flags) { if ((nmemb >= MUL_NO_OVERFLOW || size >= MUL_NO_OVERFLOW) && nmemb > 0 && SIZE_MAX / nmemb < size) { if (flags & M_CANFAIL) return (NULL); panic("mallocarray: overflow %zu * %zu", nmemb, size); } return (malloc(size * nmemb, type, flags)); } From owner-freebsd-arch@freebsd.org Mon Feb 1 20:14:40 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B4675A9857D for ; Mon, 1 Feb 2016 20:14:40 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id A3A4D14B0 for ; Mon, 1 Feb 2016 20:14:40 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id A0AFBA9857C; Mon, 1 Feb 2016 20:14:40 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0550A9857B for ; Mon, 1 Feb 2016 20:14:40 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 80B1814AE for ; Mon, 1 Feb 2016 20:14:40 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 8C2CEB946; Mon, 1 Feb 2016 15:14:39 -0500 (EST) From: John Baldwin To: Jilles Tjoelker Cc: arch@freebsd.org Subject: Re: Refactoring asynchronous I/O Date: Mon, 01 Feb 2016 12:14:28 -0800 Message-ID: <4485804.9PRGDHMp0I@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.2-STABLE; KDE/4.14.3; amd64; ; ) In-Reply-To: <20160131230214.GA37435@stack.nl> References: <2793494.0Z1kBV82mT@ralph.baldwin.cx> <20160131230214.GA37435@stack.nl> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 01 Feb 2016 15:14:39 -0500 (EST) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 20:14:40 -0000 On Monday, February 01, 2016 12:02:14 AM Jilles Tjoelker wrote: > On Tue, Jan 26, 2016 at 05:39:03PM -0800, John Baldwin wrote: > > Note that binding the AIO support to a new fileop does mean that the AIO code > > now becomes mandatory (rather than optional). We could perhaps make the > > system calls continue to be optional if people really need that, but the guts > > of the code will now need to always be in the kernel. > > Enabling this by default is OK with me as long as the easy ways to get a > stuck process are at least disabled by default. Currently, a process > gets stuck forever if it has an AIO request from or to a pipe that will > never complete. An AIO daemon should not be allowed to perform an > unbounded sleep such as for a pipe (NFS server should be OK). Mmm, I don't currently fix this for pipes, but my changes do fix this for sockets (right now if you queue multiple reads for a socket both are woken up when data arrives and if the data only satifies the first read request, the second will block an AIO daemon forever). However, having fo_aio_queue() should make this fixable for pipes as they could use their own queueing logic and handling function. It may be that pipes need to work more like sockets (where the handling is object-centric rather than request-centric, so a pipe would queue a task when it was ready and would drain any requests queued to that pipe). Pipes could either share the same AIO daemon pool as sockets or use a private pool. I'd probably like to avoid an explosion of daemon pools though. I considered having a single AIO pool, but it's kind of messy to keep the per-process job limits in the request-centric pool while also permitting object-centric handlers on the internal job queue. OTOH, if enough backends end up using object-centric handlers then the job limits might end up meaningless and we could just drop them. -- John Baldwin From owner-freebsd-arch@freebsd.org Mon Feb 1 20:46:28 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A5CE6A98F31 for ; Mon, 1 Feb 2016 20:46:28 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 888878DF for ; Mon, 1 Feb 2016 20:46:28 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 85BA0A98F30; Mon, 1 Feb 2016 20:46:28 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8551CA98F2F for ; Mon, 1 Feb 2016 20:46:28 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-qk0-f182.google.com (mail-qk0-f182.google.com [209.85.220.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C7A98DE for ; Mon, 1 Feb 2016 20:46:28 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-qk0-f182.google.com with SMTP id s68so56502951qkh.3 for ; Mon, 01 Feb 2016 12:46:28 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=sVH7zQrwKpXnUATtAtwp/O1VZbeuzKYtTJsOLmZEfqw=; b=H0KjmGEJq8rb5vUcdmSAh65MPd2rPGPm0ge70rLt+4TfkwRvPXO3FWv8I0zOOea+9z zll6YQz1dAs3SbhfWmoWrrUZXc6AXSowRxCME9eStT13LmM7cj3yFpuvWXpq/6mYKHJm oIlqJJLjuG/dL6EmKXLCfsuprUrr4AxF0Nny6m2ITEE7BDHn5Bq319KL9eF9HMDiPikk sDPUAzPzy8luo8kNcem7pcg+7I5ICW2qm/pDkie/Eo4RG/Ab7HtfAEeRXY7TTkBCWLZv Mi1QR4yUx3567JDzSuthfR6iTs931F68yyUsijeTwrtirUtMvG5lLZ6NJKh4qhCcn4HA zayQ== X-Gm-Message-State: AG10YORl8T14jr7MLQ46jLsOQ3/Bn2WVA4eMelmvq2Qxw6Xo1rPbSiW5nYKIojDWHdFtIg== X-Received: by 10.55.82.2 with SMTP id g2mr30936854qkb.53.1454357817160; Mon, 01 Feb 2016 12:16:57 -0800 (PST) Received: from mail-yk0-f179.google.com (mail-yk0-f179.google.com. [209.85.160.179]) by smtp.gmail.com with ESMTPSA id d188sm13807625qkb.9.2016.02.01.12.16.56 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 01 Feb 2016 12:16:57 -0800 (PST) Received: by mail-yk0-f179.google.com with SMTP id u9so31697704ykd.1 for ; Mon, 01 Feb 2016 12:16:56 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.37.83.133 with SMTP id h127mr13804102ybb.115.1454357816631; Mon, 01 Feb 2016 12:16:56 -0800 (PST) Reply-To: cem@FreeBSD.org Received: by 10.37.4.23 with HTTP; Mon, 1 Feb 2016 12:16:56 -0800 (PST) In-Reply-To: References: Date: Mon, 1 Feb 2016 12:16:56 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: OpenBSD mallocarray From: Conrad Meyer To: C Turt Cc: arch@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 20:46:28 -0000 On Mon, Feb 1, 2016 at 11:57 AM, C Turt wrote: > I've recently started browsing the OpenBSD kernel source code, and have > found the mallocarray function positively wonderful. I would like to > discuss the possibility of getting this into FreeBSD kernel. > > ... > > The mallocarray is a wrapper for malloc which can be used in this > situations to detect an integer overflow before allocating: Sure. +1 from me. I don't think we want the M_CANFAIL hack, though. Best, Conrad From owner-freebsd-arch@freebsd.org Mon Feb 1 20:56:21 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C68AEA9728B for ; Mon, 1 Feb 2016 20:56:21 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id A8323C83 for ; Mon, 1 Feb 2016 20:56:21 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id A5787A9728A; Mon, 1 Feb 2016 20:56:21 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A51B5A97289 for ; Mon, 1 Feb 2016 20:56:21 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-io0-x236.google.com (mail-io0-x236.google.com [IPv6:2607:f8b0:4001:c06::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 70F44C82; Mon, 1 Feb 2016 20:56:21 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-io0-x236.google.com with SMTP id 9so93146839iom.1; Mon, 01 Feb 2016 12:56:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=08w5IKqT8wpvfbvVivFxVwsd/5Af15Ij9urJPqAd9PE=; b=PCYOXE9792AjZy35BuGrUeCz96orGZjD5/HUC3QhxSr6HOdfAjwOC2ufrTWkJBXX/I ca6PO8kBSzn6wyt2bg1XJ/p9Q5S2gzQ4J+hSMzfWPijxhB5KPHJMsZ2bglHxN3mlVuht YjoBzsLwRxNMkbCSQ9+Evonn8W0GfpAyhyYNIMM5nIyxDOkxWVwiMvvHUNzCHlhZS1kW e0GcIiwa5OWRA923225xZmZ8Fjk+jOzbFe7PXYxwFLwhFYnLffJtrqbdXpY5jbkjC14n 3CIr1MukNdhR9fotlcvTUSbk31EHEH0X3H3NoHhldeoCPYFfoq78DcHpB3za5oqjAzjk zJRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=08w5IKqT8wpvfbvVivFxVwsd/5Af15Ij9urJPqAd9PE=; b=T/P/RmaNI6fMerF30MjWVoSFeBZNprghfvaPCOWNwo0KFV/jfaT7gFrsfhAabf8n9K 48BnAmj14xIhqbIgjN8/qn9vWMlWSG34SF1HAQNwzXp+kx/EJzKRLk9JtB/OyyEjo/yY Ca/8AnAlybPbg/r8bnJCnU/mzVC4EFr/M4S7bAXzGiGOnTE8sC/9mLLwL0da+LhX3OKe +eAh7IhNfR7xHZO/ffekt+6FNG12JNESFa2Jwl0ZPQ25bs6+kTDVNyQq5lMaja6doX8F eguO4RWpZzYXte1VtJvxSpx80U4EypVddjThlwT0uMFfTjYvYEE7U4t8z0zS/pSmMRR6 +blQ== X-Gm-Message-State: AG10YOT3Jl9NddgTNhWepudBPnc+6i1XSu2d04dd/0Ev3XvP5GNYyZjV5tRbdAbHOhtz4A7Ob2xbmU2alWv36g== MIME-Version: 1.0 X-Received: by 10.107.129.89 with SMTP id c86mr25577244iod.102.1454360180934; Mon, 01 Feb 2016 12:56:20 -0800 (PST) Received: by 10.107.157.6 with HTTP; Mon, 1 Feb 2016 12:56:20 -0800 (PST) In-Reply-To: References: Date: Mon, 1 Feb 2016 15:56:20 -0500 Message-ID: Subject: Re: OpenBSD mallocarray From: Ryan Stone To: cem@freebsd.org Cc: C Turt , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 20:56:22 -0000 On Mon, Feb 1, 2016 at 3:16 PM, Conrad Meyer wrote: > > Sure. +1 from me. I don't think we want the M_CANFAIL hack, though. > > Best, > Conrad > > That may be the OpenBSD equivalent of M_NOWAIT. From owner-freebsd-arch@freebsd.org Mon Feb 1 21:03:02 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CBF3FA975E8 for ; Mon, 1 Feb 2016 21:03:02 +0000 (UTC) (envelope-from mike@belopuhov.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id AB2081048 for ; Mon, 1 Feb 2016 21:03:02 +0000 (UTC) (envelope-from mike@belopuhov.com) Received: by mailman.ysv.freebsd.org (Postfix) id AA9CFA975E7; Mon, 1 Feb 2016 21:03:02 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA3D7A975E6 for ; Mon, 1 Feb 2016 21:03:02 +0000 (UTC) (envelope-from mike@belopuhov.com) Received: from mail-wm0-x22b.google.com (mail-wm0-x22b.google.com [IPv6:2a00:1450:400c:c09::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5A1EF1045 for ; Mon, 1 Feb 2016 21:03:02 +0000 (UTC) (envelope-from mike@belopuhov.com) Received: by mail-wm0-x22b.google.com with SMTP id l66so90321646wml.0 for ; Mon, 01 Feb 2016 13:03:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=belopuhov-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=XvRHdKUFUyn41vYqbFYOjYfffquw/PHcPdR5nhNroVg=; b=oWLjHsv+TeElxdNCNksNGzFc0pdXPSTO5qOzoC3owVcMk+If3M7TTU9ZrI4v3qC0Ml q95bIk6sd4tU5sSMje7ojbqUDXzuJxe9TGQtdLciGPkX2oh8t+6wh0LbKIUABVmKXTou Dux+Bo+E9VBey+CbC9YxNoAQ2Pg5K+kj7DO0LNVQ1L/+t0+K7GWzDxky2UI9ppog7TJZ N5IMWenMjE7GlPZVbhEtOnhvd/zhjXuiVMPZFn1UeZNuGRDjlyVV3PWUR4a2JkFK0ZA6 sCNoymFRxqQVY9zbLgxAVPdGNJExoYUf1ZwJoRmPaYDBGiTsDKxt2c5fDrtnG6HUQuL7 C+nA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=XvRHdKUFUyn41vYqbFYOjYfffquw/PHcPdR5nhNroVg=; b=m3KTNiQHR9qgKomsn4Z88dg4qGuBAbeu78GEei4dBxSuo6mrV3X/vTepaaX3BZUhwk VCVEm/6ouQdnsH+18psSS6UbhR8iYCJP7hy8IMpb8LaLTGKtONlb6tb/SgiD8PsizXrO 28prA7iz8JWsu5Qpdh86/0aITunkNqPlHIGV8oa4/Npce04WjKBuf0CSS1Saor04Vb5T WDmdQSRL34l6kglVRCs4SnydrXz3fgkW9fnhsCPcRoknnGFYdnR+nGpcCZNywTQSJ4xQ Nmd710nopHl3Q3/0SwH6TVK9Bkty7PaBLho3C2zV1X8bS2nLBA8yEH7lYs/39T2lusoZ dvtQ== X-Gm-Message-State: AG10YOQDcj7sQRxR5i26uSU4QeGq9z9WomPD3kBhV7BoII3IMhTiasIHO1SY1fjYGEoaMQ== X-Received: by 10.28.129.139 with SMTP id c133mr14215034wmd.30.1454360580458; Mon, 01 Feb 2016 13:03:00 -0800 (PST) Received: from localhost (a89-182-179-216.net-htp.de. [89.182.179.216]) by smtp.gmail.com with ESMTPSA id u69sm13229843wmu.20.2016.02.01.13.02.59 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 01 Feb 2016 13:02:59 -0800 (PST) Date: Mon, 1 Feb 2016 22:02:58 +0100 From: Mike Belopuhov To: Ryan Stone Cc: "freebsd-arch@freebsd.org" Subject: Re: OpenBSD mallocarray Message-ID: <20160201210256.GA29188@yamori.belopuhov.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 21:03:03 -0000 On Mon, Feb 01, 2016 at 15:56 -0500, Ryan Stone wrote: > On Mon, Feb 1, 2016 at 3:16 PM, Conrad Meyer wrote: > > > > > Sure. +1 from me. I don't think we want the M_CANFAIL hack, though. > > > > Best, > > Conrad > > > > > That may be the OpenBSD equivalent of M_NOWAIT. Not quite. From the man page: M_CANFAIL In the M_WAITOK case, if not enough memory is available, return NULL instead of calling panic(9). If mallocarray() detects an overflow or malloc() detects an excessive allocation, return NULL instead of calling panic(9). Cheers, Mike From owner-freebsd-arch@freebsd.org Mon Feb 1 21:12:21 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E970EA9788C for ; Mon, 1 Feb 2016 21:12:21 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id C4DC913D7 for ; Mon, 1 Feb 2016 21:12:21 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: by mailman.ysv.freebsd.org (Postfix) id C43F2A9788B; Mon, 1 Feb 2016 21:12:21 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C2DD8A9788A for ; Mon, 1 Feb 2016 21:12:21 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-oi0-x235.google.com (mail-oi0-x235.google.com [IPv6:2607:f8b0:4003:c06::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8476F13D5 for ; Mon, 1 Feb 2016 21:12:21 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: by mail-oi0-x235.google.com with SMTP id p187so98594213oia.2 for ; Mon, 01 Feb 2016 13:12:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :message-id:references:to; bh=hjZQ4PIu/SklDUtTtt2FT+xKrQdSHco5e3W2W/dzfG8=; b=t4+++yk6QzGEZD22GbrqzZvK890V3nijm3uAGGqBxc5X4XVKTbM86Iwdge2qnJRmVw Sosj/DZ9Ksu2piAEMJjUtF0wGDi07CeyJG4fHRPq5Dd+J+KoGfFgOibL1fJzZeCG0OP/ jdFN49UJAzyWK1b1FHiC/1qSSPNCywrTS4FWHIgq3uNhRfaz8Y7PZoSkB5S6LP1oWURb EnQKb2QUAeBFRJtRlHQy5/0xwLyO2ejG60Z8z+MeWzvmPO2BJ+pROI4urbhqwGcRJ+Qu 7pjmu2/wyoEE8EwYK4ROfNa+05tCiuLUoJStB7H3eYufB73fs8kmJAI0hhylcSAvNE8B ySVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:message-id:references:to; bh=hjZQ4PIu/SklDUtTtt2FT+xKrQdSHco5e3W2W/dzfG8=; b=MbHufMRQKCVEptMz0kOhSQy/bYSJ6ZfH+28blCUPckTcYDYZXnD+T/0BpO+eZs+hbh MUw2G2RkeCUqbJ00+PHJcUoIFrcsL3lAlPjieooY0QAQb9VMxiZt1jCLOQtoFWwnpTFF oEqh4SngC20BzcavIoYC3uNxZLwe3JU261rAS4CWhyefIl062uyjP5H8s8ZDnPRtKOSc 0E4siBdbk5uSNS0YAIqbQpCtWxWlvVUNhFiPFyZPv1yUsy2YZgMyX0Tte5q3EF2FEdC3 Jq1+XhdeJaGPDZ0xiBR9vdOSz+z4QHwfcOykLdSRMUPBZV5n2xju7zsPtAJjSdRL77Ww PEQQ== X-Gm-Message-State: AG10YOSF6gXDFvFGqpCgGLCXeifko2fuQVuMnUSYeXuNYnuDIYyT9ilzCfhwpeySFNWCWg== X-Received: by 10.202.222.7 with SMTP id v7mr18627740oig.79.1454361140606; Mon, 01 Feb 2016 13:12:20 -0800 (PST) Received: from netflix-mac-wired.bsdimp.com ([50.253.99.174]) by smtp.gmail.com with ESMTPSA id u142sm1664489oia.19.2016.02.01.13.12.19 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 01 Feb 2016 13:12:20 -0800 (PST) Sender: Warner Losh Subject: Re: OpenBSD mallocarray Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Content-Type: multipart/signed; boundary="Apple-Mail=_06A5EA6B-7610-4B5E-89C1-F2DF5947F9BF"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5.2 From: Warner Losh In-Reply-To: <20160201210256.GA29188@yamori.belopuhov.com> Date: Mon, 1 Feb 2016 14:12:20 -0700 Cc: Ryan Stone , "freebsd-arch@freebsd.org" Message-Id: <1EA0ECF5-D7AC-430E-957D-C4D49F9A872B@bsdimp.com> References: <20160201210256.GA29188@yamori.belopuhov.com> To: Mike Belopuhov X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 21:12:22 -0000 --Apple-Mail=_06A5EA6B-7610-4B5E-89C1-F2DF5947F9BF Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Feb 1, 2016, at 2:02 PM, Mike Belopuhov wrote: >=20 > On Mon, Feb 01, 2016 at 15:56 -0500, Ryan Stone wrote: >> On Mon, Feb 1, 2016 at 3:16 PM, Conrad Meyer wrote: >>=20 >>>=20 >>> Sure. +1 from me. I don't think we want the M_CANFAIL hack, = though. >>>=20 >>> Best, >>> Conrad >>>=20 >>>=20 >> That may be the OpenBSD equivalent of M_NOWAIT. >=20 > Not quite. =46rom the man page: >=20 > M_CANFAIL >=20 > In the M_WAITOK case, if not enough memory is available, > return NULL instead of calling panic(9). If mallocarray() > detects an overflow or malloc() detects an excessive > allocation, return NULL instead of calling panic(9). Yea, we don=E2=80=99t want it calling panic. Ever. That turns an = overflow into a DoS. Arguments should be properly checked so we can properly return EINVAL for bat-**** crazy ones. FreeBSD=E2=80=99s malloc doesn=E2=80=99t cave an excessive detector in it. My concern with this is that we have a number of different allocation routines in FreeBSD. This only goes after the malloc() vector, and even then it requires code changes. At best, CANFAIL is a kludge to fail with a panic instead of an overflow. That=E2=80=99s got to be at most a transient thing until all = the code that it is kludged into with out proper thought is fixed. I=E2=80=99m= not sure that=E2=80=99s something that we want to encourage. I=E2=80=99m all = for safety, but this flag seems both unsafe and unwise. Warner --Apple-Mail=_06A5EA6B-7610-4B5E-89C1-F2DF5947F9BF Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJWr8o1AAoJEGwc0Sh9sBEA0I4QAK8rwoVmnywbLqxT5DZBarWY FE5UYxFB0DBd39wqGLFcoQ/zISL9rzXeAR2lKVOCS+FZE2lLxgYMACXaqIQrPaE2 KU4qFUNbcI1W9MOtf4oVOvLxAcGZNeAWl1/DJVILsh1YuBG8bKddTejIZSzuyQbE JxqCqpm47Tu+HF7og24IOUIN4SD9ko241NPS9tL0W0GU7ka4YodufK1khGFs303l SrkvXtTmkgxsAgR5jXOwyUThEuhnky1GQg++kOT2WjFJQ3fnpgdziDRrzMe0m2Dj x4YusPQkwpf8ydSO7MuIIFjiTw12eGg4AqQXWVit7bZ0I2+tSl9QZKt9e6tlgskU hcsGVR2395NU0zS1CZJGzqb5aNXJIozYOES8ZcDX272DLhack1IX0whH/hHb+RC7 z8AY0QYn28QrcUf40XI8QN25Y1V5Optn5QJCCIzylbt9Rat8orNvsH0tfLIA7uj/ M1Ak7IcLOkbJ8ioKSON26yd4tE1NzuzB6YeuD4NHVGWX47WvjzTnJlgK8pr1ZYUJ qvh3mqf/K7o+XlAUBzb8bNPN0VYH0LFUHWLYWHYJbvbaB+Ud7zu+yZbLdil9g7rC awXMK2GQ4D0DiK7UZSnMzR0Vjh0V0SYMdTvUfGhaZGIrAvlFARL9rADyav7D/mBB ozWxYbeOH8l3/XmWlJt9 =NecL -----END PGP SIGNATURE----- --Apple-Mail=_06A5EA6B-7610-4B5E-89C1-F2DF5947F9BF-- From owner-freebsd-arch@freebsd.org Mon Feb 1 21:00:30 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4ED2EA973C8 for ; Mon, 1 Feb 2016 21:00:30 +0000 (UTC) (envelope-from ezekielmotta11@gmail.com) Received: from mail-lf0-x22b.google.com (mail-lf0-x22b.google.com [IPv6:2a00:1450:4010:c07::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CF023D9E for ; Mon, 1 Feb 2016 21:00:29 +0000 (UTC) (envelope-from ezekielmotta11@gmail.com) Received: by mail-lf0-x22b.google.com with SMTP id l143so38068206lfe.2 for ; Mon, 01 Feb 2016 13:00:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=2H4GgoIeSmOpu/+3Wlc84g51eaVc2OEkjaUep4hY+WQ=; b=rsyd5NnZiyXfJvkxAXVZ/e1Xojs35Qa7HEXKFPO5PyUOlNaLCRo2v3+BChUQh82ufm xNvcZJThwpiiPdIvwZ56XOvGAgXmBO6LZObvNvAJhMZRX1PpTiMQW0nsYYjO81j1sdW7 wUs6u4ji9HZkZwZ0zPXEzW/vdiI0CWs69d8KLCVWXDcOwwO/NaSLIuO36Jxli89EhZ1i 3+EHVnW0eL9Kn3rea1kkYo8W7FtuOoptnJ2R3AolVuhDTMt78dAEyL3OYzdEPjR/7+rr WwaUHqiFP47eev4UJNM8XmpxzGRy5+/N2CubesDFKDYI0CHpgEtidie5hBaFv1rnjrvb BkPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=2H4GgoIeSmOpu/+3Wlc84g51eaVc2OEkjaUep4hY+WQ=; b=NwkRUsxDNatzE+6HHRcSB/E4pjIVgDRqCCokmxoLJ2Qw3lKGJQ238Ri0SXgBypvUMG roHytgxkkSjW71CjHJfUOv8JVIYxlq/OCe5YNuy5Gw93T5RrVaAt5lrCqlvZpfPTnzVv 88/oHGnQ8O6tQ3KHgsXDYqN9czyThxaei3Ba8BL+joG0/j7XFQJe6tJPUUjHYNSNO6Fd c4NV/FsYNim+bq6XVkGnMmQEzzKrlxXJv5cHrayQjr5ZJo91DL3T8sD3XxrVaQ9Iio4+ vjVrhskN5BqPFQLSC3y5XvvrL+AhRF69YQJ3GxNXrYt/rTsdq8iPhWRGsMR2VaVpOF8/ RkZQ== X-Gm-Message-State: AG10YOSf6YSY8V4f7y7UaupEd055NM9MPk0dLjpmJQ8EtU6wGiqdhJnJUhZdu4myRitxhRmI7ogjMI43PV1O2g== MIME-Version: 1.0 X-Received: by 10.25.160.1 with SMTP id j1mr9773743lfe.35.1454360427439; Mon, 01 Feb 2016 13:00:27 -0800 (PST) Received: by 10.25.210.16 with HTTP; Mon, 1 Feb 2016 13:00:27 -0800 (PST) Date: Mon, 1 Feb 2016 20:00:27 -0100 Message-ID: Subject: info From: ezekiel motta To: freebsd-arch@FreeBSD.org X-Mailman-Approved-At: Mon, 01 Feb 2016 21:41:49 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 21:00:30 -0000 Hello I have asked a friend to build a FreeBSD friendly computer for me. I wish to know which processors, ssd's etc are best suited for a FreeBSD desktop. I am not a technical person. Could you please recommend a page where i can find this info. kind regards zeke From owner-freebsd-arch@freebsd.org Mon Feb 1 22:41:37 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BADADA7682F for ; Mon, 1 Feb 2016 22:41:37 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-lf0-x22b.google.com (mail-lf0-x22b.google.com [IPv6:2a00:1450:4010:c07::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 44FE9891 for ; Mon, 1 Feb 2016 22:41:37 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mail-lf0-x22b.google.com with SMTP id l143so39579443lfe.2 for ; Mon, 01 Feb 2016 14:41:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=b0coZZyI653FxYQOjxMFyK6jdONocCtN1aeJ1jzdE7s=; b=JICvUyGmjPPd5i75REzFykkT67pBpVyCKEEJyl1pUCwGdVLlwwAWQuD/XU3hT5brGR OQCm0qCoUuXtjBvHZ+rdPLC9lN/CwYHQdkExcIjKXxpBMgNxMi0ZRFjh12pu7VHKEOvj xP1tvhvTfCBbOTPGd1sJxSZwyPYwuiDuJxz2c2/D5isC691hG7X/NywaO2aDbJaqj2QI 64gpeGnW9bCUDWhwResIS8vnMH2T4liDxNZKnj1gP9tI6q3N5b0PSavwb/ZSvvgwsOd3 Sa8DhLl29nv15LBcIq+pIKbVBHb2WBVZ/bIBH0lqBEH1Mn+CCoSScIhdb7mNJZrfTz/M 1JSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=b0coZZyI653FxYQOjxMFyK6jdONocCtN1aeJ1jzdE7s=; b=QzxKwNXHUzxBAJGryqeGBnbkgY3BLETjZ9bXbQsqe3LwqHxhFUps4SlYqJ/aGJ1CYP 5+w6CDuO+4yELmJd3PtiP+1MsDrg7MRU8scVyLFhS0l4YdJ9525/9dnS0dKDeb20Flb/ SWmd/2yc641+vGukqWBgtkBcfXXPXqprE8YVhL6urg/QZQrciEQCWdCTM+jznCTwAStz ODqkpyrHFiGvWnUApuE6VjIznrx7GHTO5CXyEBpWWQ0Bc9AgM3F3wpgrhgocXgsUkEup U/jMCkme40CeXn244OY7sS0KAj9DLkHLUWGyLf8HRzViXI7g1Hrh+mF3DYt49MI4wSJ3 RFHQ== X-Gm-Message-State: AG10YORLgEYOdrTWfYZmb9l17W/x5JOkWkdTINlAzwsaSxS327rZZ+0XZuXKvk/HIWL6B7+URqcCYKAUHSNK3w== MIME-Version: 1.0 X-Received: by 10.25.162.202 with SMTP id l193mr9591220lfe.138.1454366495445; Mon, 01 Feb 2016 14:41:35 -0800 (PST) Received: by 10.112.160.133 with HTTP; Mon, 1 Feb 2016 14:41:35 -0800 (PST) In-Reply-To: References: Date: Mon, 1 Feb 2016 14:41:35 -0800 Message-ID: Subject: Re: info From: NGie Cooper To: ezekiel motta Cc: "freebsd-arch@FreeBSD.org Arch" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 22:41:37 -0000 On Mon, Feb 1, 2016 at 1:00 PM, ezekiel motta wrote: > Hello > > I have asked a friend to build a FreeBSD friendly computer for me. > > I wish to know which processors, ssd's etc are best suited for a FreeBSD > desktop. > > I am not a technical person. Could you please recommend a page where i can > find this info. Zeke, I believe this question is better suited for the freebsd-questions@ list or maybe the forums under the System Hardware section [1]? This mailing list is mainly for FreeBSD software architecture discussion. That being said, there's a laptop page that might be helpful [2], and if I were you I might trawl the PCBSD forums a bit -- there might be some helpful info there [3]. Hope this helps! -NGie 1. https://forums.freebsd.org/forums/system-hardware.32/ 2. https://wiki.freebsd.org/Laptops 3. https://forums.pcbsd.org/ From owner-freebsd-arch@freebsd.org Mon Feb 1 22:49:01 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 837AAA76A76 for ; Mon, 1 Feb 2016 22:49:01 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 6809DB2A for ; Mon, 1 Feb 2016 22:49:01 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: by mailman.ysv.freebsd.org (Postfix) id 650D8A76A75; Mon, 1 Feb 2016 22:49:01 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4ACFDA76A74 for ; Mon, 1 Feb 2016 22:49:01 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2974EB29 for ; Mon, 1 Feb 2016 22:49:00 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id 8A1505A9F0E; Mon, 1 Feb 2016 22:48:54 +0000 (UTC) Date: Mon, 1 Feb 2016 22:48:54 +0000 From: Brooks Davis To: Warner Losh Cc: Mike Belopuhov , "freebsd-arch@freebsd.org" , Ryan Stone Subject: Re: OpenBSD mallocarray Message-ID: <20160201224854.GB1747@spindle.one-eyed-alien.net> References: <20160201210256.GA29188@yamori.belopuhov.com> <1EA0ECF5-D7AC-430E-957D-C4D49F9A872B@bsdimp.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Kj7319i9nmIyA2yE" Content-Disposition: inline In-Reply-To: <1EA0ECF5-D7AC-430E-957D-C4D49F9A872B@bsdimp.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 22:49:01 -0000 --Kj7319i9nmIyA2yE Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Feb 01, 2016 at 02:12:20PM -0700, Warner Losh wrote: >=20 > > On Feb 1, 2016, at 2:02 PM, Mike Belopuhov wrote: > >=20 > > On Mon, Feb 01, 2016 at 15:56 -0500, Ryan Stone wrote: > >> On Mon, Feb 1, 2016 at 3:16 PM, Conrad Meyer wrote: > >>=20 > >>>=20 > >>> Sure. +1 from me. I don't think we want the M_CANFAIL hack, though. > >>>=20 > >>> Best, > >>> Conrad > >>>=20 > >>>=20 > >> That may be the OpenBSD equivalent of M_NOWAIT. > >=20 > > Not quite. From the man page: > >=20 > > M_CANFAIL > >=20 > > In the M_WAITOK case, if not enough memory is available, > > return NULL instead of calling panic(9). If mallocarray() > > detects an overflow or malloc() detects an excessive > > allocation, return NULL instead of calling panic(9). >=20 > Yea, we don???t want it calling panic. Ever. That turns an overflow > into a DoS. Arguments should be properly checked so we can > properly return EINVAL for bat-**** crazy ones. FreeBSD???s malloc > doesn???t cave an excessive detector in it. >=20 > My concern with this is that we have a number of different allocation > routines in FreeBSD. This only goes after the malloc() vector, and > even then it requires code changes. >=20 > At best, CANFAIL is a kludge to fail with a panic instead of an > overflow. That???s got to be at most a transient thing until all the > code that it is kludged into with out proper thought is fixed. I???m not > sure that???s something that we want to encourage. I???m all for > safety, but this flag seems both unsafe and unwise. Given that current code could be doing literally anything in the overflow case (and thanks to modern undefined behavior optimizations is likely doing something extraordinarily bizarre), I think turning overflows into panics is a good thing. Yes, you can argue that means you've added a DoS vector, but best case you had an under allocation and bizarre memory corruption before. If the default or even only behavior is going to be that overflow fails then we need a static checker that ensure we check the return value even in the M_WAITOK. Otherwise there will be blind conversions of malloc to mallocarray that go unchecked. -- Brooks --Kj7319i9nmIyA2yE Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJWr+DVAAoJEKzQXbSebgfAigwH/RythIPti+fjj3fDQsduTQNU BhCCM4q7tsIKAF6hpZ5Vu87N6riJynYLCoAwHX904z9ehrVz/iiVSJPUZU+i7Skz DK2ExoehWRLbaJb/Hd/Io2brPEgi7RszccLQebH2ZprJ/6UjJqithE04mtPU99ve cGZK4PFf++zDZUL5i6gVSszABPM7MgdsKSiTObBl0GDZmgftdvMSF1IJlIMXbDww DgfU0eIbql6SdI7ki+46g36ZLWHw2wlbz/kZf4HaNcHMjPhf4lbvA6ScT0uo3rYZ OyW5Q0wUO66KnplvRGXX+rz9a9VAjwUAwOGm/urGn/9ogowxt+9OgY8e/6iC6LM= =jbJ2 -----END PGP SIGNATURE----- --Kj7319i9nmIyA2yE-- From owner-freebsd-arch@freebsd.org Mon Feb 1 22:49:15 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9FB9A76AA4 for ; Mon, 1 Feb 2016 22:49:15 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 8721FBD4 for ; Mon, 1 Feb 2016 22:49:15 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mailman.ysv.freebsd.org (Postfix) id 8634DA76AA3; Mon, 1 Feb 2016 22:49:15 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6BEC2A76AA2 for ; Mon, 1 Feb 2016 22:49:15 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qg0-x22c.google.com (mail-qg0-x22c.google.com [IPv6:2607:f8b0:400d:c04::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2AE06BD0 for ; Mon, 1 Feb 2016 22:49:15 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qg0-x22c.google.com with SMTP id e32so133856424qgf.3 for ; Mon, 01 Feb 2016 14:49:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=rG0tATVzdVStVwa66H0C/D70vGa4mPk9e8x6mibKmok=; b=RHem74BTnBuc1/Ws+wb60zysrBnyBuYGwQ4Am4vKvD+A83hyFr2AIW/nEsK0Zld9Vh H7CMDoFt5wWhgmIosSAJKpUN6dU/LNiXsffxJQw47LbkbFhvJmhCr6VVdbRE8z76jq3p XzZVpNptbBILJKItGN2OHJa8nepAy7hw8o4HffYaAgSV2g1CsWdy7CTDcMugziHUNEyb DKKgMf8RTh2MlT305tsOvorXLnMfXvQxGTaN+V0xRJ6uY5WBOB8LBv9EDsTjk6Th1ny0 kxoBvqytBQteNR899cloa77Q5tG1JWFRTJ+fG9oTlJUQbl0QPGvlTrxGb3snhxX0zmZB vrAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=rG0tATVzdVStVwa66H0C/D70vGa4mPk9e8x6mibKmok=; b=SKt25/U0gxZXEV5xlWL3Caz+3zwoxPfMPT6c5mMx2cFKsO5dOtt1r5N7hjLbX/nJX0 QC+PhhoBTHsair2fnFiw5+K00v6n61uUdNbp206mrS2nd8G0X/sR/BgdIOzKxnlGRL/C d/8oNYVvFI1XRp/fyStup9SI7feWXRE5asIobvcPGZxPYi3SQdDinOWKSunqFn4K1f8s fyj3VKTRsi6iezQl89J0uQDiKLlV7QdKaM1MQPzFmUxvmq2+BSzTu4a5ihVQlZvu7DNn nB+9aJa8rGsxeK5AY5/sZ5TueSQZ3VGTykcGxrv/mpYRjBZ1sDKWxc8ICp/r0EqKf4AO t88w== X-Gm-Message-State: AG10YORITBsoy5qi2V3fNXa8CTvcbB3LgwEw0IllrrKLe4dIpjVdxZki4ZmcgZAquutXugqu+fwpcqrjPMaJfg== MIME-Version: 1.0 X-Received: by 10.140.229.209 with SMTP id z200mr15870589qhb.40.1454366954194; Mon, 01 Feb 2016 14:49:14 -0800 (PST) Sender: wlosh@bsdimp.com Received: by 10.140.30.166 with HTTP; Mon, 1 Feb 2016 14:49:14 -0800 (PST) X-Originating-IP: [69.53.245.11] In-Reply-To: References: <20160201210256.GA29188@yamori.belopuhov.com> <1EA0ECF5-D7AC-430E-957D-C4D49F9A872B@bsdimp.com> Date: Mon, 1 Feb 2016 15:49:14 -0700 X-Google-Sender-Auth: ybhq0qN_BYbnKYVn7_A7IAMf3xQ Message-ID: Subject: Re: OpenBSD mallocarray From: Warner Losh To: cem@freebsd.org Cc: Mike Belopuhov , "freebsd-arch@freebsd.org" , Ryan Stone Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 22:49:15 -0000 On Mon, Feb 1, 2016 at 3:19 PM, Conrad Meyer wrote: > On Mon, Feb 1, 2016 at 1:12 PM, Warner Losh wrote: > > > >> On Feb 1, 2016, at 2:02 PM, Mike Belopuhov wrote: > >> Not quite. From the man page: > >> > >> M_CANFAIL > >> > >> In the M_WAITOK case, if not enough memory is available, > >> return NULL instead of calling panic(9). If mallocarray() > >> detects an overflow or malloc() detects an excessive > >> allocation, return NULL instead of calling panic(9). > > > > Yea, we don=E2=80=99t want it calling panic. Ever. That turns an overfl= ow > > into a DoS. > > I disagree. The panic is essentially an assertion that malloc was > passed valid arguments. We have similar invariants assertions > throughout the kernel and it is the only sane thing to do with > overflow + M_WAITOK. M_WAITOK callers today will do something equally > stupid if they get a NULL result from mallocarray(). We only do this for things that can't happen. If the user can trigger this panic by passing some bogus, unchecked value that doesn't get checked until here, that's bad. That's fundamentally different than getting 'freeing free inode' from ufs. Users can't trigger the latter. > > > Arguments should be properly checked > > Yes! That's why the assertion is a good thing. We disagree on this then. This isn't a 'fail safe' this is a 'fail with denial of system'. that' > > At best, CANFAIL is a kludge to fail with a panic instead of an > > overflow. > > No, that's backwards. In CANFAIL mode, mallocarray returns NULL > instead of panicing immediately. It's a kludge so the caller doesn't > have to do overflow checking. Then it's a horrible interface. We failed badly with the MPSAFE interface. It was OK at first, but then when everybody uses it, then it because obvious that it was the wrong decision. > > That=E2=80=99s got to be at most a transient thing until all the > > code that it is kludged into with out proper thought is fixed. > > You mean the panic? What fallback behavior would you prefer? If the > caller requested an overflowing allocation, there really isn't > anything sane to do besides immediately panic (again, for M_WAITOK). > Even M_NOWAIT callers may or may not do something sane. > I'd prefer that we return NULL. Let the caller decide the policy, not some stupid thing down in the bowls of malloc because I forgot to include some stupid new flag. M_NOWAIT callers know to check for NULL, or they have a bug. It would be the same for mallocarray: you must check for NULL and unwind if you get that back. There's no need for the CANFAIL flag and it's a horrible API. Warner From owner-freebsd-arch@freebsd.org Mon Feb 1 23:01:15 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC246A76E5D for ; Mon, 1 Feb 2016 23:01:15 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 89EE11009 for ; Mon, 1 Feb 2016 23:01:15 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mailman.ysv.freebsd.org (Postfix) id 86E5DA76E5A; Mon, 1 Feb 2016 23:01:15 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 857E5A76E57 for ; Mon, 1 Feb 2016 23:01:15 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qg0-x234.google.com (mail-qg0-x234.google.com [IPv6:2607:f8b0:400d:c04::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3810C1004 for ; Mon, 1 Feb 2016 23:01:15 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qg0-x234.google.com with SMTP id 6so133075382qgy.1 for ; Mon, 01 Feb 2016 15:01:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=OrU5oez5BDzlrehKZkGmTDVcJ/cILZRct8TU+DjbzvE=; b=Gmf7qa4RrB/E+BPj1b55Gzve03cnxWaKnzNQKnTqnQD2gkLjFvkUOsq12zFQ53JGNU cK6McxUZjfRCWCRBJlJ7i/nwzM6Y1KE6mKbyavwCF//DYKpTuylEghrdjWRLmBAEEYXD uHyF8BP6cEF1zf601tMedcP6dvL0+4scHEuDXXpEeREQJInfhHmahDp6jR0Jvy5robzh DrFY5qtXP5TxTCPUdOEFAOC3BH9VI6V3ybP+W7pyCiUcyzMM9i41sPeyXtPMqcqG3c50 Ruu2Gv+8fA37pBSkeOcvZAgAT6trnRu8sybJUdsWjmrxO7jfQc/NfwhSouyCllKVcXkX 3Tcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=OrU5oez5BDzlrehKZkGmTDVcJ/cILZRct8TU+DjbzvE=; b=lutOoNDmZsg7MNkQbHmmT0MbMNpnffGV6o63IkA1rqSJ1ENIWfL/5CLJNWVoK3uwmP a7HFKcnNHXbWDjqVwv7WTl+IKMFQ1YGpm6aoT+FSn2lIzjPWeFVr//wjWrWjTYMQHE3+ ZsZ6R9qv5dubAvS8SfVlX8bR8SogxW8YtBw/uC0gvgnxWRxXpL5OYV0SkMT64GVOS1pv Vsj8qXarQoF+nvgMTMWzYzI56hVLFvQHEOt3EQW80qniz448ie1pV2maQduVO63ZUApl H5PDqKnMOM5AWhgoBRsn22w5LEjAMG/sAdK/YDaNdL9Oi5JFm2hTRQcb2491sRQqGgAw OkYA== X-Gm-Message-State: AG10YOSA4OdLuUtGA3SNmCXxunMLtf5uBO+XrzZfQlJci3rehwl4gE71D9L3B7eWpO1b1qvQ+uml7hww/5YmyA== MIME-Version: 1.0 X-Received: by 10.140.99.53 with SMTP id p50mr31336915qge.97.1454367674361; Mon, 01 Feb 2016 15:01:14 -0800 (PST) Sender: wlosh@bsdimp.com Received: by 10.140.30.166 with HTTP; Mon, 1 Feb 2016 15:01:14 -0800 (PST) X-Originating-IP: [69.53.245.11] In-Reply-To: <20160201224854.GB1747@spindle.one-eyed-alien.net> References: <20160201210256.GA29188@yamori.belopuhov.com> <1EA0ECF5-D7AC-430E-957D-C4D49F9A872B@bsdimp.com> <20160201224854.GB1747@spindle.one-eyed-alien.net> Date: Mon, 1 Feb 2016 16:01:14 -0700 X-Google-Sender-Auth: M8YEVlymbk--hESawu_aU7fUITM Message-ID: Subject: Re: OpenBSD mallocarray From: Warner Losh To: Brooks Davis Cc: Mike Belopuhov , "freebsd-arch@freebsd.org" , Ryan Stone Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 23:01:15 -0000 On Mon, Feb 1, 2016 at 3:48 PM, Brooks Davis wrote: > On Mon, Feb 01, 2016 at 02:12:20PM -0700, Warner Losh wrote: > > > > > On Feb 1, 2016, at 2:02 PM, Mike Belopuhov wrote: > > > > > > On Mon, Feb 01, 2016 at 15:56 -0500, Ryan Stone wrote: > > >> On Mon, Feb 1, 2016 at 3:16 PM, Conrad Meyer wrote: > > >> > > >>> > > >>> Sure. +1 from me. I don't think we want the M_CANFAIL hack, though. > > >>> > > >>> Best, > > >>> Conrad > > >>> > > >>> > > >> That may be the OpenBSD equivalent of M_NOWAIT. > > > > > > Not quite. From the man page: > > > > > > M_CANFAIL > > > > > > In the M_WAITOK case, if not enough memory is available, > > > return NULL instead of calling panic(9). If mallocarray() > > > detects an overflow or malloc() detects an excessive > > > allocation, return NULL instead of calling panic(9). > > > > Yea, we don???t want it calling panic. Ever. That turns an overflow > > into a DoS. Arguments should be properly checked so we can > > properly return EINVAL for bat-**** crazy ones. FreeBSD???s malloc > > doesn???t cave an excessive detector in it. > > > > My concern with this is that we have a number of different allocation > > routines in FreeBSD. This only goes after the malloc() vector, and > > even then it requires code changes. > > > > At best, CANFAIL is a kludge to fail with a panic instead of an > > overflow. That???s got to be at most a transient thing until all the > > code that it is kludged into with out proper thought is fixed. I???m not > > sure that???s something that we want to encourage. I???m all for > > safety, but this flag seems both unsafe and unwise. > > Given that current code could be doing literally anything in the > overflow case (and thanks to modern undefined behavior optimizations is > likely doing something extraordinarily bizarre), I think turning overflows > into panics is a good thing. Yes, you can argue that means you've added > a DoS vector, but best case you had an under allocation and bizarre > memory corruption before. If the default or even only behavior is going > to be that overflow fails then we need a static checker that ensure we > check the return value even in the M_WAITOK. Otherwise there will be > blind conversions of malloc to mallocarray that go unchecked. > Returning NULL should be sufficient. Blind conversion of malloc to mallocarray in the kernel is also stupid. Intelligent conversion is needed to ensure that the error conditions are handled correctly. There's no need for a flag to say 'I am going to do the right thing if you give me NULL back'. The conversion should do the right thing when you get NULL back. A quick survey of the current kernel shows that there's not very many that could be using user defined values, but does show a large number of places where we could use this API. I guess this comes down to 'why is it an unreasonable burden to test the return value in code that's converted?' We're already changing the code. If you absolutely must have a flag, I'd prefer M_CANPANIC or something that is also easy to add for the 'mindless' case that we can easily grep for so we know when we're removed all the stupid 'mindless' cases from the tree. Warner From owner-freebsd-arch@freebsd.org Mon Feb 1 23:24:56 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4DB71A9776B for ; Mon, 1 Feb 2016 23:24:56 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 2C7541E51 for ; Mon, 1 Feb 2016 23:24:56 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 29237A9776A; Mon, 1 Feb 2016 23:24:56 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 28BBEA97769 for ; Mon, 1 Feb 2016 23:24:56 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-qg0-f47.google.com (mail-qg0-f47.google.com [209.85.192.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E0BED1E4F for ; Mon, 1 Feb 2016 23:24:55 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-qg0-f47.google.com with SMTP id o11so133670014qge.2 for ; Mon, 01 Feb 2016 15:24:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=WUGwnb9oStHioOVVgtOoBkst6Z1nwBDs0girDJYaNoI=; b=Wnkd7rzy4nMBH4AgFNuPqbzhiv5avkBxvOJQJuziOxPmMNzXr/Iio5Iinb5qjmd0RO hScD3HIt/B7vFgYPa/WmyGRbT2wTtLuC62FQ9Odmpx0vKaubpWYoS5orZPW7+O6MQqU5 a1TfPbFi6bknzJvartjvkkaqjbl7naQsxyf2uk2VET52Rswc/kE8rVNHIlHxxr7iXPZ+ c7b/chxm7sCHp7PBVNFzpQvG/aq168q1urNoQwf91ogwCryCE/sndqT6S2RoaYtPV805 qCeUlNR+rwoWgar/6mtBOEwsGVV4j9POajPRWA1Vkd6lC4nxlqRW1WyZDJTpZL+p00SJ QOsg== X-Gm-Message-State: AG10YOR5qV52IQ1ZWovH9a3uB8etB+XH1laVETzhcrUnyqoofmmLhEYcr4Tu4IyXMoiE7w== X-Received: by 10.140.28.66 with SMTP id 60mr31424754qgy.74.1454367458018; Mon, 01 Feb 2016 14:57:38 -0800 (PST) Received: from mail-yk0-f171.google.com (mail-yk0-f171.google.com. [209.85.160.171]) by smtp.gmail.com with ESMTPSA id n203sm4957672qhc.43.2016.02.01.14.57.37 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 01 Feb 2016 14:57:37 -0800 (PST) Received: by mail-yk0-f171.google.com with SMTP id z13so65632599ykd.0 for ; Mon, 01 Feb 2016 14:57:37 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.37.64.198 with SMTP id n189mr6104480yba.124.1454367457260; Mon, 01 Feb 2016 14:57:37 -0800 (PST) Reply-To: cem@FreeBSD.org Received: by 10.37.2.132 with HTTP; Mon, 1 Feb 2016 14:57:37 -0800 (PST) In-Reply-To: References: <20160201210256.GA29188@yamori.belopuhov.com> <1EA0ECF5-D7AC-430E-957D-C4D49F9A872B@bsdimp.com> Date: Mon, 1 Feb 2016 14:57:37 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: OpenBSD mallocarray From: Conrad Meyer To: Warner Losh Cc: Mike Belopuhov , "freebsd-arch@freebsd.org" , Ryan Stone Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 23:24:56 -0000 On Mon, Feb 1, 2016 at 2:49 PM, Warner Losh wrote: > On Mon, Feb 1, 2016 at 3:19 PM, Conrad Meyer wrote: >> On Mon, Feb 1, 2016 at 1:12 PM, Warner Losh wrote: >> > Yea, we don=E2=80=99t want it calling panic. Ever. That turns an overf= low >> > into a DoS. >> >> I disagree. The panic is essentially an assertion that malloc was >> passed valid arguments. We have similar invariants assertions >> throughout the kernel and it is the only sane thing to do with >> overflow + M_WAITOK. M_WAITOK callers today will do something equally >> stupid if they get a NULL result from mallocarray(). > > We only do this for things that can't happen. Exactly. malloc requests that overflow a size_t *cannot* happen. Today they are undefined behavior / at best memory corruption. This is not something we want users to be able to trigger, ever. > If the user can trigger this > panic by passing some bogus, unchecked value that doesn't get checked > until here, that's bad. Agreed. > That's fundamentally different than getting > 'freeing free inode' from ufs. Users can't trigger the latter. Users must not be able to trigger this panic either. > We disagree on this then. This isn't a 'fail safe' this is a 'fail > with denial of system'. that' What is your alternative proposal, in this scenario, that results in any better result than DoS? Heap corruption and code execution is not a better result than DoS, IMO. Keep in mind that if the user controls array size allocation in this scenario, even without the panic they may DoS the system with a huge-but-smaller-than-size_t kernel memory allocation. >> You mean the panic? What fallback behavior would you prefer? If the >> caller requested an overflowing allocation, there really isn't >> anything sane to do besides immediately panic (again, for M_WAITOK). >> Even M_NOWAIT callers may or may not do something sane. > > I'd prefer that we return NULL. Let the caller decide the policy, > not some stupid thing down in the bowls of malloc because > I forgot to include some stupid new flag. If we don't panic explicitly, we panic implicitly for unfixed M_WAITOK users of the interface. I think the explicit panic is better, at least until callers are fixed. > M_NOWAIT callers know to check for NULL, or they have a bug. Yes, but it's a different error. E.g. callers cannot handle EAGAIN like EI= NVAL. > It would be the same for mallocarray: you must check for NULL and > unwind if you get that back. There's no need for the CANFAIL flag > and it's a horrible API. I agree CANFAIL is terrible but come down on the opposite side for the default behavior when given overflowing arguments. Overflowing arguments should never happen. It is a bogus use of the API. Best, Conrad From owner-freebsd-arch@freebsd.org Mon Feb 1 23:58:11 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9DB27A98400 for ; Mon, 1 Feb 2016 23:58:11 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 7DFC0F01 for ; Mon, 1 Feb 2016 23:58:11 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 7C3BEA983FF; Mon, 1 Feb 2016 23:58:11 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 61FD6A983FE for ; Mon, 1 Feb 2016 23:58:11 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-qg0-f48.google.com (mail-qg0-f48.google.com [209.85.192.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 27603EFF for ; Mon, 1 Feb 2016 23:58:10 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-qg0-f48.google.com with SMTP id u30so117450qge.1 for ; Mon, 01 Feb 2016 15:58:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=Rsi6PYpYTUqRsy5M8h+1cwm+aDsFud5jvScRXkwIPZs=; b=EowAesd1lwgl4S2Kb4zoU4KKPAQLLv6QMTWr0qXy9kDob3GZ/ckp9OInKFjKMohcBo hGlkKFRgEtcReJuW040fSr4YPucUwkK63XniWVAYhSNtPCXjDfxbQDSBPvfngQOU3kCu 4AtLspx+mKB4DBrSj+X5XeMiXmQXeaeI6vwsxuBycGOORCDOKz8/rS9EOHqoPEycpwis Aa+GLM3LI+BeN76EbybIMwyq+9X7/rZx/LJ/IkqeMFymJc3SSXJnAJU5pWAEzOiAdKgz 39AAg1IXquE3225adsmI8AOMPDjxik9+nQHB3cvkCOU5p6nm5Besfvupb6SEzYbHS7N3 gSuw== X-Gm-Message-State: AG10YOQytZ6u55Hy0EeoCcxwb8maihW8MHwSkB37IJiy0CZcHXUwPTUDivGXvRnWZrLr/w== X-Received: by 10.140.81.80 with SMTP id e74mr31505217qgd.27.1454365176888; Mon, 01 Feb 2016 14:19:36 -0800 (PST) Received: from mail-yk0-f177.google.com (mail-yk0-f177.google.com. [209.85.160.177]) by smtp.gmail.com with ESMTPSA id s64sm14993090qgd.28.2016.02.01.14.19.36 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 01 Feb 2016 14:19:36 -0800 (PST) Received: by mail-yk0-f177.google.com with SMTP id z13so64837556ykd.0 for ; Mon, 01 Feb 2016 14:19:35 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.37.102.70 with SMTP id a67mr14640230ybc.89.1454365175883; Mon, 01 Feb 2016 14:19:35 -0800 (PST) Reply-To: cem@FreeBSD.org Received: by 10.37.4.23 with HTTP; Mon, 1 Feb 2016 14:19:35 -0800 (PST) In-Reply-To: <1EA0ECF5-D7AC-430E-957D-C4D49F9A872B@bsdimp.com> References: <20160201210256.GA29188@yamori.belopuhov.com> <1EA0ECF5-D7AC-430E-957D-C4D49F9A872B@bsdimp.com> Date: Mon, 1 Feb 2016 14:19:35 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: OpenBSD mallocarray From: Conrad Meyer To: Warner Losh Cc: Mike Belopuhov , "freebsd-arch@freebsd.org" , Ryan Stone Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 23:58:11 -0000 On Mon, Feb 1, 2016 at 1:12 PM, Warner Losh wrote: > >> On Feb 1, 2016, at 2:02 PM, Mike Belopuhov wrote: >> Not quite. From the man page: >> >> M_CANFAIL >> >> In the M_WAITOK case, if not enough memory is available, >> return NULL instead of calling panic(9). If mallocarray() >> detects an overflow or malloc() detects an excessive >> allocation, return NULL instead of calling panic(9). > > Yea, we don=E2=80=99t want it calling panic. Ever. That turns an overflow > into a DoS. I disagree. The panic is essentially an assertion that malloc was passed valid arguments. We have similar invariants assertions throughout the kernel and it is the only sane thing to do with overflow + M_WAITOK. M_WAITOK callers today will do something equally stupid if they get a NULL result from mallocarray(). > Arguments should be properly checked Yes! That's why the assertion is a good thing. > At best, CANFAIL is a kludge to fail with a panic instead of an > overflow. No, that's backwards. In CANFAIL mode, mallocarray returns NULL instead of panicing immediately. It's a kludge so the caller doesn't have to do overflow checking. > That=E2=80=99s got to be at most a transient thing until all the > code that it is kludged into with out proper thought is fixed. You mean the panic? What fallback behavior would you prefer? If the caller requested an overflowing allocation, there really isn't anything sane to do besides immediately panic (again, for M_WAITOK). Even M_NOWAIT callers may or may not do something sane. Best, Conrad From owner-freebsd-arch@freebsd.org Mon Feb 1 23:59:58 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D91ABA98472 for ; Mon, 1 Feb 2016 23:59:58 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id C4374FC0 for ; Mon, 1 Feb 2016 23:59:58 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: by mailman.ysv.freebsd.org (Postfix) id C0531A98471; Mon, 1 Feb 2016 23:59:58 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BFE98A98470 for ; Mon, 1 Feb 2016 23:59:58 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 997F8FBF for ; Mon, 1 Feb 2016 23:59:58 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id 43F325A9F27; Mon, 1 Feb 2016 23:59:57 +0000 (UTC) Date: Mon, 1 Feb 2016 23:59:57 +0000 From: Brooks Davis To: Warner Losh Cc: Mike Belopuhov , "freebsd-arch@freebsd.org" , Ryan Stone Subject: Re: OpenBSD mallocarray Message-ID: <20160201235957.GC1747@spindle.one-eyed-alien.net> References: <20160201210256.GA29188@yamori.belopuhov.com> <1EA0ECF5-D7AC-430E-957D-C4D49F9A872B@bsdimp.com> <20160201224854.GB1747@spindle.one-eyed-alien.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ADZbWkCsHQ7r3kzd" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 23:59:58 -0000 --ADZbWkCsHQ7r3kzd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Feb 01, 2016 at 04:01:14PM -0700, Warner Losh wrote: > On Mon, Feb 1, 2016 at 3:48 PM, Brooks Davis wrote: >=20 > > On Mon, Feb 01, 2016 at 02:12:20PM -0700, Warner Losh wrote: > > > > > > > On Feb 1, 2016, at 2:02 PM, Mike Belopuhov wro= te: > > > > > > > > On Mon, Feb 01, 2016 at 15:56 -0500, Ryan Stone wrote: > > > >> On Mon, Feb 1, 2016 at 3:16 PM, Conrad Meyer wro= te: > > > >> > > > >>> > > > >>> Sure. +1 from me. I don't think we want the M_CANFAIL hack, tho= ugh. > > > >>> > > > >>> Best, > > > >>> Conrad > > > >>> > > > >>> > > > >> That may be the OpenBSD equivalent of M_NOWAIT. > > > > > > > > Not quite. From the man page: > > > > > > > > M_CANFAIL > > > > > > > > In the M_WAITOK case, if not enough memory is available, > > > > return NULL instead of calling panic(9). If mallocarray() > > > > detects an overflow or malloc() detects an excessive > > > > allocation, return NULL instead of calling panic(9). > > > > > > Yea, we don???t want it calling panic. Ever. That turns an overflow > > > into a DoS. Arguments should be properly checked so we can > > > properly return EINVAL for bat-**** crazy ones. FreeBSD???s malloc > > > doesn???t cave an excessive detector in it. > > > > > > My concern with this is that we have a number of different allocation > > > routines in FreeBSD. This only goes after the malloc() vector, and > > > even then it requires code changes. > > > > > > At best, CANFAIL is a kludge to fail with a panic instead of an > > > overflow. That???s got to be at most a transient thing until all the > > > code that it is kludged into with out proper thought is fixed. I???m = not > > > sure that???s something that we want to encourage. I???m all for > > > safety, but this flag seems both unsafe and unwise. > > > > Given that current code could be doing literally anything in the > > overflow case (and thanks to modern undefined behavior optimizations is > > likely doing something extraordinarily bizarre), I think turning overfl= ows > > into panics is a good thing. Yes, you can argue that means you've added > > a DoS vector, but best case you had an under allocation and bizarre > > memory corruption before. If the default or even only behavior is going > > to be that overflow fails then we need a static checker that ensure we > > check the return value even in the M_WAITOK. Otherwise there will be > > blind conversions of malloc to mallocarray that go unchecked. > > >=20 > Returning NULL should be sufficient. Blind conversion of malloc to > mallocarray in the kernel is also stupid. Intelligent conversion is > needed to ensure that the error conditions are handled correctly. > There's no need for a flag to say 'I am going to do the right thing > if you give me NULL back'. The conversion should do the right > thing when you get NULL back. A quick survey of the current kernel > shows that there's not very many that could be using user defined > values, but does show a large number of places where we could > use this API. On further consideration, I think returning NULL is sufficient since most of the time failure to check will result in a nearby fault. The main thing is eliminating the undefined behavior. -- Brooks --ADZbWkCsHQ7r3kzd Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJWr/F8AAoJEKzQXbSebgfAJacH/RJ0Rvp/1/JRqguNmnsJgJ1e TzWHmqeir6d1aN2jEQ+bDWvVSAp8v7SB074o7cuz6dX6neyA5O1+qeTwCNFbemM7 6Ii5JB1UVOlTA0VCb9FM8CFhw816bg94phGL+RGLOO499P3uUq2ybVeSDD8ns54x D+0jifE+WGs94zdZhcFMr+CQIVqKGD8yGVa4xj1cgJLD6hOUlw/yxjfvXD1ZLezL y15H6F0hRC98lrBYRCaDNLEqhpLvBH5zq5B+cuADQzmpJYk40HrsSypLmXlTyrWF zpb9elq93ohk73uGdo5jXeafoJGiWSQtSKW7C1Gp3AZsMleHanrM0dWIVk/vSgg= =gN2r -----END PGP SIGNATURE----- --ADZbWkCsHQ7r3kzd-- From owner-freebsd-arch@freebsd.org Tue Feb 2 06:05:22 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 55BA0A9824B for ; Tue, 2 Feb 2016 06:05:22 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 3D57A15C7 for ; Tue, 2 Feb 2016 06:05:22 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: by mailman.ysv.freebsd.org (Postfix) id 3BFBCA98249; Tue, 2 Feb 2016 06:05:22 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3B871A98248 for ; Tue, 2 Feb 2016 06:05:22 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail110.syd.optusnet.com.au (mail110.syd.optusnet.com.au [211.29.132.97]) by mx1.freebsd.org (Postfix) with ESMTP id DEEFD15C6; Tue, 2 Feb 2016 06:05:21 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c211-30-166-197.carlnfd1.nsw.optusnet.com.au (c211-30-166-197.carlnfd1.nsw.optusnet.com.au [211.30.166.197]) by mail110.syd.optusnet.com.au (Postfix) with ESMTPS id 43BF17848BB; Tue, 2 Feb 2016 17:05:18 +1100 (AEDT) Date: Tue, 2 Feb 2016 17:05:18 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Warner Losh cc: cem@freebsd.org, Mike Belopuhov , Ryan Stone , "freebsd-arch@freebsd.org" Subject: Re: OpenBSD mallocarray In-Reply-To: Message-ID: <20160202160137.M916@besplex.bde.org> References: <20160201210256.GA29188@yamori.belopuhov.com> <1EA0ECF5-D7AC-430E-957D-C4D49F9A872B@bsdimp.com> MIME-Version: 1.0 X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=PfoC/XVd c=1 sm=1 tr=0 a=KA6XNC2GZCFrdESI5ZmdjQ==:117 a=L9H7d07YOLsA:10 a=9cW_t1CCXrUA:10 a=s5jvgZ67dGcA:10 a=nlC_4_pT8q9DhB4Ho9EA:9 a=6I5d2MoRAAAA:8 a=7Qk2ozbKAAAA:8 a=zHAF4flPilWhpgNLD_wA:9 a=pFdfRmNuPZVy2IBl:21 a=gRaiNp1KNaO4jvP0:21 a=45ClL6m2LaAA:10 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2016 06:05:22 -0000 On Mon, 1 Feb 2016, Warner Losh wrote: > On Mon, Feb 1, 2016 at 3:19 PM, Conrad Meyer wrote: > >> On Mon, Feb 1, 2016 at 1:12 PM, Warner Losh wrote: > ... >>> That=E2=80=99s got to be at most a transient thing until all the >>> code that it is kludged into with out proper thought is fixed. >> >> You mean the panic? What fallback behavior would you prefer? If the >> caller requested an overflowing allocation, there really isn't >> anything sane to do besides immediately panic (again, for M_WAITOK). >> Even M_NOWAIT callers may or may not do something sane. > > I'd prefer that we return NULL. Let the caller decide the policy, > not some stupid thing down in the bowls of malloc because > I forgot to include some stupid new flag. > > M_NOWAIT callers know to check for NULL, or they have a bug. > It would be the same for mallocarray: you must check for NULL and > unwind if you get that back. There's no need for the CANFAIL flag > and it's a horrible API. The whole API is horrible. It is a wrapper that is intended to make things simpler, but actually makes them more complicated. It is even worse than calloc(3) since normal use of malloc(9) is to never fail, while both malloc(3) and calloc(3) always have failure modes. It is the array size calculation that can overflow, not the allocation. First I note how using calloc(3) is only slightly more complicated unless sloppy error handling is acceptable. Good version that does checks up front like most malloc(9) users already do, but for malloc(3). =09if (size !=3D 0 && SIZE_MAX / size < num) =09=09return (EINVAL); =09if (malloc((size_t)size * num) =3D=3D NULL) =09=09laboriously_handle_error_usually_worse_than_null_ptr_core(); The error "can't happen", and we can make that even more likely by changing SIZE_MAX to a small value. We should usually change SIZE_MAX to a small value anyway to limit denials of service to other caleers. The overflow check can be hidden in a macro (not in an inline function without expanding everything to uintmax_t), but using the macro isn't much easier: #define=09MALLOC_ARRAY_CHECK(num, size, limit) \ =09((size) =3D=3D 0 || limit / (size) >=3D (num)) =2E.. =09if (!MALLOC_ARRAY_CHECK(num, size, SIZE_MAX)) =09=09return (EINVAL); =09if (malloc((size_t)size * num) =3D=3D NULL) =09=09laboriously_handle_error_usually_worse_than_null_ptr_core(); Sloppy code using calloc(): =09if (calloc(num, size) =3D=3D NULL) =09=09laboriously_handle_error_usually_worse_than_null_ptr_core(); Equivalent code using calloc(): =09if (calloc(num, size) =3D=3D NULL) { =09=09/* =09=09 * calloc() doesn't tell us the precise reason for failure, =09=09 * and decoding errno wouldn't be much better than the =09=09 * following, =09=09 * so if our API is not as bad as that then we must check =09=09 * for overflow like we should have done up front. =09=09 */ =09=09if (!MALLOC_ARRAY_CHECK(num, size, SIZE_MAX)) =09=09=09return (EINVAL); =09=09laboriously_handle_error_usually_worse_than_null_ptr_core(); =09} Using malloc(9) with M_NOWAIT is similar except the error handling must be even more laborious to be correct. It usually isn't correct, but is more needed and is sometimes better than a null pointer panic. Using M_WAITOK, we lose the simpler code that is possible by not having error handling in every caller. Good version: =09if (!MALLOC_ARRAY_CHECK(num, size, my_limit)) =09=09return (EINVAL); =09ptr =3D malloc((size_t)size * num, M_FOO, M_WAITOK); The error handling is obviously neither neither laborious or incorrect. It is just to handle a DOS from callers. Worse version: put all the args in a function. Add flags to control the error handling. I forget the details of mallocarray(). The above can be done by adding 2 args: =09ptr =3D mymallocarray(num, size, sizelimit, M_FOO, M_WAITOK); =09if (ptr =3D=3D NULL) =09=09return (EINVAL); This is actually a couple of words less complicated, but also less clear. It hides the limit check. Don't forget to expand the types taken by the function to uintmax_t. Otherwise, overflow may occur when the function is called when the type of num, size of sizelimit is larger than size_t (most likely if it is a 64-bit type on a 32-bit arch). The macro handles this automatically provided size_max <=3D SIZE_MAX. Bruce From owner-freebsd-arch@freebsd.org Wed Feb 3 19:39:29 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 79613A9AE53; Wed, 3 Feb 2016 19:39:29 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5351B1F3E; Wed, 3 Feb 2016 19:39:29 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id DB9B7B917; Wed, 3 Feb 2016 14:39:27 -0500 (EST) From: John Baldwin To: freebsd-arch@freebsd.org Cc: Warner Losh , Brooks Davis , Mike Belopuhov , Ryan Stone , "freebsd-arch@freebsd.org" Subject: Re: OpenBSD mallocarray Date: Wed, 03 Feb 2016 11:39:11 -0800 Message-ID: <1955470.jNCaThvui8@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.2-STABLE; KDE/4.14.3; amd64; ; ) In-Reply-To: References: <20160201224854.GB1747@spindle.one-eyed-alien.net> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 03 Feb 2016 14:39:28 -0500 (EST) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Feb 2016 19:39:29 -0000 On Monday, February 01, 2016 04:01:14 PM Warner Losh wrote: > On Mon, Feb 1, 2016 at 3:48 PM, Brooks Davis wrote: > > > On Mon, Feb 01, 2016 at 02:12:20PM -0700, Warner Losh wrote: > > > > > > > On Feb 1, 2016, at 2:02 PM, Mike Belopuhov wrote: > > > > > > > > On Mon, Feb 01, 2016 at 15:56 -0500, Ryan Stone wrote: > > > >> On Mon, Feb 1, 2016 at 3:16 PM, Conrad Meyer wrote: > > > >> > > > >>> > > > >>> Sure. +1 from me. I don't think we want the M_CANFAIL hack, though. > > > >>> > > > >>> Best, > > > >>> Conrad > > > >>> > > > >>> > > > >> That may be the OpenBSD equivalent of M_NOWAIT. > > > > > > > > Not quite. From the man page: > > > > > > > > M_CANFAIL > > > > > > > > In the M_WAITOK case, if not enough memory is available, > > > > return NULL instead of calling panic(9). If mallocarray() > > > > detects an overflow or malloc() detects an excessive > > > > allocation, return NULL instead of calling panic(9). > > > > > > Yea, we don???t want it calling panic. Ever. That turns an overflow > > > into a DoS. Arguments should be properly checked so we can > > > properly return EINVAL for bat-**** crazy ones. FreeBSD???s malloc > > > doesn???t cave an excessive detector in it. > > > > > > My concern with this is that we have a number of different allocation > > > routines in FreeBSD. This only goes after the malloc() vector, and > > > even then it requires code changes. > > > > > > At best, CANFAIL is a kludge to fail with a panic instead of an > > > overflow. That???s got to be at most a transient thing until all the > > > code that it is kludged into with out proper thought is fixed. I???m not > > > sure that???s something that we want to encourage. I???m all for > > > safety, but this flag seems both unsafe and unwise. > > > > Given that current code could be doing literally anything in the > > overflow case (and thanks to modern undefined behavior optimizations is > > likely doing something extraordinarily bizarre), I think turning overflows > > into panics is a good thing. Yes, you can argue that means you've added > > a DoS vector, but best case you had an under allocation and bizarre > > memory corruption before. If the default or even only behavior is going > > to be that overflow fails then we need a static checker that ensure we > > check the return value even in the M_WAITOK. Otherwise there will be > > blind conversions of malloc to mallocarray that go unchecked. > > > > Returning NULL should be sufficient. Blind conversion of malloc to > mallocarray in the kernel is also stupid. Intelligent conversion is > needed to ensure that the error conditions are handled correctly. > There's no need for a flag to say 'I am going to do the right thing > if you give me NULL back'. The conversion should do the right > thing when you get NULL back. A quick survey of the current kernel > shows that there's not very many that could be using user defined > values, but does show a large number of places where we could > use this API. > > I guess this comes down to 'why is it an unreasonable burden to > test the return value in code that's converted?' We're already changing > the code. > > If you absolutely must have a flag, I'd prefer M_CANPANIC or something > that is also easy to add for the 'mindless' case that we can easily > grep for so we know when we're removed all the stupid 'mindless' > cases from the tree. Having M_WAITOK-anything return NULL will be a POLA violation. It doesn't return NULL for anything else. I think having a separate M_CANFAIL flag is also rather pointless. If we want to have this, I think it should work similar to malloc(). M_WAITOK panics if you do stupid things (malloc(9) does this for sufficiently large overflow when it exhausts kmem contrary to Warner's earlier claim), M_NOWAIT returns NULL. In general I think I most prefer Bruce's approach of having a separate macro to check for overflow explicitly so it can be handled as a separate error. In particular, if mallocarry(..., M_NOWAIT) fails, is it because of memory shortage (in which case retrying in the future might be sensible) or is it due to overflow (in which case it will fail no matter how many times you retry)? You'd then have to have the macro anyway to differentiate and handle this case. Warner also seems to be assuming that we should do check for overflow explicitly for any user-supplied values before calling malloc() or malloc()-like things. This means N hand-rolled (and possibly buggy) checks, or a shared macro to do the check. I think this is another argument in favor of Bruce's approach. :) If you go that route, then mallocarray() is really just an assertion checker. It should only fail because a programmer ommitted an explicit overflow check for a user-supplied value or screwed up an in-kernel value. In that case I think panic'ing sooner when the overflow is obvious is more useful for debugging the error than a NULL pointer deference some time later (or requests that get retried forever and go possibly unnoticed). -- John Baldwin From owner-freebsd-arch@freebsd.org Wed Feb 3 19:43:56 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68DB8A9B08B for ; Wed, 3 Feb 2016 19:43:56 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qg0-x233.google.com (mail-qg0-x233.google.com [IPv6:2607:f8b0:400d:c04::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 21918636 for ; Wed, 3 Feb 2016 19:43:56 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qg0-x233.google.com with SMTP id b35so24024562qge.0 for ; Wed, 03 Feb 2016 11:43:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=jNeuAhO9iljZYJiMEgErI/xYOwJIwl7MVLzs9IO0hzU=; b=KgCJ/u70s81z8G3wS2xLoegl1SZ4Fo9Hgq3FkOaKnUc5r73XWaeUTJ+fc/cXky2cGW J53pDZz+CTqdDqH1fBKz0lziytiNcvs6C62f4ei1Xh5o6bTUEZZDDPBdPm/P3DUJufXd /JOMp1kJF7xjhRgBVJiROFwd8nNrT7jDVXUGOx2a0cLlC75cSK/r/KLKhd2AX0Heng1n G0JRQv+Ji/w0MeoCVONrphDndJ9MsYoagveQ12kJqBrPFdo2jDI+iMMkRljU/MPeN8jR K0MgJvAbzPAHRJQ05WOG7E2LO9ZuS10Ivu8zQbvUlapVuJDkuWt8iVqODTJzU7IScOlk KbQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=jNeuAhO9iljZYJiMEgErI/xYOwJIwl7MVLzs9IO0hzU=; b=cJqKGhgwnJXZfetddiWSQ5WgGLsAm1sL9jMSv+46NgPh2+Zj5dbp2UUv3VlfBsnF08 ud/tXJIAVYtHKZuI6eTvPove7lx72iESYyK3gZe5NoX3Z5I6xycpCOHEWGkWRhH0WLJM pZJflzAXVw9il14tknFPEm+OXmdEvoplk1w4pZBxj+PRf31NlnnGTvbaLG36J06gdK9g kyKYg9QPSxuZp2hz6oYrMOBr/DVlCmEalnyCv/9fr8tRGFoD5QifIoi71RsGsy6AG0lc AEqxHoCqQ8Bup7tswxG2W8fghep1ABE38JfOXooCxQvMBTdjwWbCFBA45yi7pinbz+zy mvrg== X-Gm-Message-State: AG10YOS7LvD0oZGO4F6vLb9LF0zWok+gVJy93JG1cYtQgg9RQGwoNN52EjHJ64wu+NVRxAFyrG12/7oaBR/kTQ== MIME-Version: 1.0 X-Received: by 10.140.93.87 with SMTP id c81mr4055675qge.46.1454528635160; Wed, 03 Feb 2016 11:43:55 -0800 (PST) Sender: wlosh@bsdimp.com Received: by 10.140.30.166 with HTTP; Wed, 3 Feb 2016 11:43:55 -0800 (PST) X-Originating-IP: [192.30.190.9] In-Reply-To: <1955470.jNCaThvui8@ralph.baldwin.cx> References: <20160201224854.GB1747@spindle.one-eyed-alien.net> <1955470.jNCaThvui8@ralph.baldwin.cx> Date: Wed, 3 Feb 2016 12:43:55 -0700 X-Google-Sender-Auth: 1N6j9mliEpAKditMJUsvenlTkH0 Message-ID: Subject: Re: OpenBSD mallocarray From: Warner Losh To: John Baldwin Cc: "freebsd-arch@freebsd.org" , Brooks Davis , Mike Belopuhov , Ryan Stone , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Feb 2016 19:43:56 -0000 On Wed, Feb 3, 2016 at 12:39 PM, John Baldwin wrote: > On Monday, February 01, 2016 04:01:14 PM Warner Losh wrote: > > On Mon, Feb 1, 2016 at 3:48 PM, Brooks Davis wrote: > > > > > On Mon, Feb 01, 2016 at 02:12:20PM -0700, Warner Losh wrote: > > > > > > > > > On Feb 1, 2016, at 2:02 PM, Mike Belopuhov > wrote: > > > > > > > > > > On Mon, Feb 01, 2016 at 15:56 -0500, Ryan Stone wrote: > > > > >> On Mon, Feb 1, 2016 at 3:16 PM, Conrad Meyer > wrote: > > > > >> > > > > >>> > > > > >>> Sure. +1 from me. I don't think we want the M_CANFAIL hack, > though. > > > > >>> > > > > >>> Best, > > > > >>> Conrad > > > > >>> > > > > >>> > > > > >> That may be the OpenBSD equivalent of M_NOWAIT. > > > > > > > > > > Not quite. From the man page: > > > > > > > > > > M_CANFAIL > > > > > > > > > > In the M_WAITOK case, if not enough memory is available, > > > > > return NULL instead of calling panic(9). If mallocarray() > > > > > detects an overflow or malloc() detects an excessive > > > > > allocation, return NULL instead of calling panic(9). > > > > > > > > Yea, we don???t want it calling panic. Ever. That turns an overflow > > > > into a DoS. Arguments should be properly checked so we can > > > > properly return EINVAL for bat-**** crazy ones. FreeBSD???s malloc > > > > doesn???t cave an excessive detector in it. > > > > > > > > My concern with this is that we have a number of different allocation > > > > routines in FreeBSD. This only goes after the malloc() vector, and > > > > even then it requires code changes. > > > > > > > > At best, CANFAIL is a kludge to fail with a panic instead of an > > > > overflow. That???s got to be at most a transient thing until all the > > > > code that it is kludged into with out proper thought is fixed. I???m > not > > > > sure that???s something that we want to encourage. I???m all for > > > > safety, but this flag seems both unsafe and unwise. > > > > > > Given that current code could be doing literally anything in the > > > overflow case (and thanks to modern undefined behavior optimizations is > > > likely doing something extraordinarily bizarre), I think turning > overflows > > > into panics is a good thing. Yes, you can argue that means you've > added > > > a DoS vector, but best case you had an under allocation and bizarre > > > memory corruption before. If the default or even only behavior is > going > > > to be that overflow fails then we need a static checker that ensure we > > > check the return value even in the M_WAITOK. Otherwise there will be > > > blind conversions of malloc to mallocarray that go unchecked. > > > > > > > Returning NULL should be sufficient. Blind conversion of malloc to > > mallocarray in the kernel is also stupid. Intelligent conversion is > > needed to ensure that the error conditions are handled correctly. > > There's no need for a flag to say 'I am going to do the right thing > > if you give me NULL back'. The conversion should do the right > > thing when you get NULL back. A quick survey of the current kernel > > shows that there's not very many that could be using user defined > > values, but does show a large number of places where we could > > use this API. > > > > I guess this comes down to 'why is it an unreasonable burden to > > test the return value in code that's converted?' We're already changing > > the code. > > > > If you absolutely must have a flag, I'd prefer M_CANPANIC or something > > that is also easy to add for the 'mindless' case that we can easily > > grep for so we know when we're removed all the stupid 'mindless' > > cases from the tree. > > Having M_WAITOK-anything return NULL will be a POLA violation. It doesn't > return NULL for anything else. I think having a separate M_CANFAIL flag > is also rather pointless. If we want to have this, I think it should > work similar to malloc(). M_WAITOK panics if you do stupid things > (malloc(9) does this for sufficiently large overflow when it exhausts kmem > contrary to Warner's earlier claim), M_NOWAIT returns NULL. > Exausting kmem isn't influenced by simple args. But I do stand corrected. > In general I think I most prefer Bruce's approach of having a separate > macro > to check for overflow explicitly so it can be handled as a separate error. > In particular, if mallocarry(..., M_NOWAIT) fails, is it because of memory > shortage (in which case retrying in the future might be sensible) or is it > due to overflow (in which case it will fail no matter how many times you > retry)? You'd then have to have the macro anyway to differentiate and > handle > this case. > > Warner also seems to be assuming that we should do check for overflow > explicitly for any user-supplied values before calling malloc() or > malloc()-like things. This means N hand-rolled (and possibly buggy) > checks, > or a shared macro to do the check. I think this is another argument in > favor > of Bruce's approach. :) > I like Bruce's approach. And it works for more than just malloc. > If you go that route, then mallocarray() is really just an assertion > checker. It should only fail because a programmer ommitted an explicit > overflow check for a user-supplied value or screwed up an in-kernel > value. In that case I think panic'ing sooner when the overflow is obvious > is more useful for debugging the error than a NULL pointer deference > some time later (or requests that get retried forever and go possibly > unnoticed). > That would be fine. On its own, mallocarray() has all the issues we've talked about. Warner From owner-freebsd-arch@freebsd.org Fri Feb 5 20:22:06 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EA5A0A77AD7 for ; Fri, 5 Feb 2016 20:22:06 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id D9031100F for ; Fri, 5 Feb 2016 20:22:06 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id D88DFA77AD6; Fri, 5 Feb 2016 20:22:06 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D73D6A77AD5 for ; Fri, 5 Feb 2016 20:22:06 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B4D27100E for ; Fri, 5 Feb 2016 20:22:06 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id BDC6EB948; Fri, 5 Feb 2016 15:22:05 -0500 (EST) From: John Baldwin To: Jilles Tjoelker Cc: arch@freebsd.org Subject: Re: Refactoring asynchronous I/O Date: Fri, 05 Feb 2016 12:21:52 -0800 Message-ID: <9227739.EqUaAQ57pU@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.2-STABLE; KDE/4.14.3; amd64; ; ) In-Reply-To: <20160131230214.GA37435@stack.nl> References: <2793494.0Z1kBV82mT@ralph.baldwin.cx> <20160131230214.GA37435@stack.nl> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 05 Feb 2016 15:22:05 -0500 (EST) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 20:22:07 -0000 On Monday, February 01, 2016 12:02:14 AM Jilles Tjoelker wrote: > On Tue, Jan 26, 2016 at 05:39:03PM -0800, John Baldwin wrote: > > Note that binding the AIO support to a new fileop does mean that the AIO code > > now becomes mandatory (rather than optional). We could perhaps make the > > system calls continue to be optional if people really need that, but the guts > > of the code will now need to always be in the kernel. > > Enabling this by default is OK with me as long as the easy ways to get a > stuck process are at least disabled by default. Currently, a process > gets stuck forever if it has an AIO request from or to a pipe that will > never complete. An AIO daemon should not be allowed to perform an > unbounded sleep such as for a pipe (NFS server should be OK). One thing I could do is split vfs_aio.c into two files: kern_aio.c that holds the "library" such as aio_aqueue() / aio_complete(), etc. and a sys_aio.c that is just the system calls. kern_aio.c would be standard, but sys_aio.c could still be optional and aio.ko would contain it. This would still make AIO optional, though aio.ko would be fairly small, so not having it probably wouldn't save much in terms of size. Does this seem reasonable or is a trivial aio.ko not worth it? -- John Baldwin From owner-freebsd-arch@freebsd.org Fri Feb 5 21:32:16 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 69C79A9E825 for ; Fri, 5 Feb 2016 21:32:16 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 594B4193A for ; Fri, 5 Feb 2016 21:32:16 +0000 (UTC) (envelope-from jilles@stack.nl) Received: by mailman.ysv.freebsd.org (Postfix) id 583E8A9E824; Fri, 5 Feb 2016 21:32:16 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 57D9DA9E823 for ; Fri, 5 Feb 2016 21:32:16 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mailhost.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 258BC1939; Fri, 5 Feb 2016 21:32:16 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from toad2.stack.nl (toad2.stack.nl [IPv6:2001:610:1108:5010::161]) by mx1.stack.nl (Postfix) with ESMTP id 8679735932D; Fri, 5 Feb 2016 22:32:12 +0100 (CET) Received: by toad2.stack.nl (Postfix, from userid 1677) id 61629892D6; Fri, 5 Feb 2016 22:32:12 +0100 (CET) Date: Fri, 5 Feb 2016 22:32:12 +0100 From: Jilles Tjoelker To: John Baldwin Cc: arch@freebsd.org Subject: Re: Refactoring asynchronous I/O Message-ID: <20160205213212.GA97435@stack.nl> References: <2793494.0Z1kBV82mT@ralph.baldwin.cx> <20160131230214.GA37435@stack.nl> <9227739.EqUaAQ57pU@ralph.baldwin.cx> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <9227739.EqUaAQ57pU@ralph.baldwin.cx> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 21:32:16 -0000 On Fri, Feb 05, 2016 at 12:21:52PM -0800, John Baldwin wrote: > On Monday, February 01, 2016 12:02:14 AM Jilles Tjoelker wrote: > > On Tue, Jan 26, 2016 at 05:39:03PM -0800, John Baldwin wrote: > > > Note that binding the AIO support to a new fileop does mean that > > > the AIO code now becomes mandatory (rather than optional). We > > > could perhaps make the system calls continue to be optional if > > > people really need that, but the guts of the code will now need to > > > always be in the kernel. > > Enabling this by default is OK with me as long as the easy ways to get a > > stuck process are at least disabled by default. Currently, a process > > gets stuck forever if it has an AIO request from or to a pipe that will > > never complete. An AIO daemon should not be allowed to perform an > > unbounded sleep such as for a pipe (NFS server should be OK). > One thing I could do is split vfs_aio.c into two files: kern_aio.c that > holds the "library" such as aio_aqueue() / aio_complete(), etc. and a > sys_aio.c that is just the system calls. kern_aio.c would be standard, > but sys_aio.c could still be optional and aio.ko would contain it. > This would still make AIO optional, though aio.ko would be fairly small, > so not having it probably wouldn't save much in terms of size. > Does this seem reasonable or is a trivial aio.ko not worth it? It is one possible option. Another option is to refuse AIO for kinds of file that have not been vetted to avoid blocking an AIO daemon for too long. "Kinds of file" is about the code that will be executed to do I/O, so that, for example, /dev/klog and /dev/ttyv0 are different kinds. Depending on whether this restriction breaks existing code, it may need to be a sysctl. I don't expect a full AIO implementation for all kinds of file any time soon, so putting AIO syscalls into a kernel module will still leave the standard system without AIO for a long time, while still paying for the footprint of the AIO code. All of this would not be a problem if PCATCH worked in AIO daemons (treat aio_cancel like a pending signal in a user process) but the last time I looked this appeared quite hard to fix. -- Jilles Tjoelker