From owner-freebsd-arch@freebsd.org Fri Dec 27 19:49:17 2019 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8AE7B1CD4E4 for ; Fri, 27 Dec 2019 19:49:17 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qv1-xf36.google.com (mail-qv1-xf36.google.com [IPv6:2607:f8b0:4864:20::f36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47ky7w5hx9z4V5t for ; Fri, 27 Dec 2019 19:49:16 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qv1-xf36.google.com with SMTP id m14so10386220qvl.3 for ; Fri, 27 Dec 2019 11:49:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=KzMs03Aq7gxS3S651YRiHs9bHgcKA85ZOVpl5CTlK8w=; b=HqBCQFZhmM8DXMXDxjF0Tsk4FN3lWjyLJ5ZWJO8Gy5sKVJA7wbHuxVdhD4RF6ZVfc4 DJju6+0bzwDCADeBR7H6Zuy6/TA0tm25oBs/dlFIl7xn4A/7UQMpW73t1XWF+wyBydmE kbhB5Gq767QCcvRdgrQ9XyJHkteG6vI2S16aPd+yAYYG9rnSlHybCiZ5h7Bb8H9esuXl RKpeDjllU26awcG+W5hVESSSWDrVCFdqyLGlVLraDuWeAZa7+J7HNjo+WM2x027ijF65 razvbYtNSbV/7x5uMgQWHIHjr5YR2BpX3yxMgvjAee9wtwkRPUl+ji2CTz4oPB7AX93v bj5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=KzMs03Aq7gxS3S651YRiHs9bHgcKA85ZOVpl5CTlK8w=; b=R9TEmsEm2DV0AJqOi3bdMLYAljhQYh4s9vjrfUlUvxCthT91IG5A1un+vbaIqXsGxA xVIoT0X+bPHSGFgyoP00NkuYqYo51YEPgFRC4ROCqt3wGWwDVpn8ykaRhHq6Q/phrorK MStzBFfJsls/w54UbHVMg46Gokxokhg9J6FLiARY0EMhLrw4Hxgzo3g4QWIvOt2RmKq2 2pOzEeaVYBEYWGJ6LCzMeRh69oFsZYyHv2e8FxTZ1Xg4e/pw9GPH1N/CYe/BuzQiQbEr q20TsFmemkQpBbc0jo4toZgBKJrBj64DNen90uEwtPaxrSirmcKFC+CvvRAnVblkEJNg KH9Q== X-Gm-Message-State: APjAAAXw67CaK4GNLfFXRaxL/ee0gy2O9mQyV+YFUqDm2XbHD49Halhg 6la7kBabAQfnRW4Tll1tIwgsCtJ1B+vaQFm5Zjiy2w== X-Google-Smtp-Source: APXvYqwxZqtkbkVPQWUTbqpJd03XT0xoLJQKdMfpHe6kLYdUWV90dANd0C6vJhWxWWFZo3bvTx2NwqLE9mcro+0DUHA= X-Received: by 2002:a05:6214:965:: with SMTP id do5mr26532700qvb.202.1577476155687; Fri, 27 Dec 2019 11:49:15 -0800 (PST) MIME-Version: 1.0 References: <6d466360-988f-e80d-3212-b6c479a5ec03@FreeBSD.org> In-Reply-To: <6d466360-988f-e80d-3212-b6c479a5ec03@FreeBSD.org> From: Warner Losh Date: Fri, 27 Dec 2019 12:49:04 -0700 Message-ID: Subject: Re: gsched: modernize or remove? To: Alexander Motin Cc: "freebsd-geom@FreeBSD.org" , "freebsd-arch@freebsd.org" , Warner Losh , Luigi Rizzo , "Andrey V. Elsukov" , Pawel Jakub Dawidek , "Conrad E. Meyer" X-Rspamd-Queue-Id: 47ky7w5hx9z4V5t X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=HqBCQFZh; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::f36) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-1.82 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; IP_SCORE(-0.82)[ipnet: 2607:f8b0::/32(-2.16), asn: 15169(-1.88), country: US(-0.05)]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arch@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCPT_COUNT_SEVEN(0.00)[8]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Dec 2019 19:49:17 -0000 On Fri, Dec 27, 2019 at 10:53 AM Alexander Motin wrote: > Hi, > > As I can see, gsched code was not really maintained for the last 10 > years since being added. It misses many GEOM features added later, such > as direct dispatch, unmapped I/O, stripesize/stripeoffset, resize, etc. > Even if some of them may require just a proper declaration, it tells me > that barely anybody used it seriously for years. But my primary concern > is the `gsched insert` implementation. Right now I got to it since it > is the last consumer of nstart/nend counters in GEOM, which I would like > to remove for performance reasons. But I also see tons of potential > problems with idea of moving providers between unaware geoms. > > So my question is: does it make sense to try fix/modernize it, or it > just be easier to remove it? Does anybody still use it, or see some > future for it? While it's kinda cool, I'm not sure it has stayed relevant enough. It's kinda at the wrong layer to do effective scheduling since the low level drivers get to pick and choose what goes to the drive when. We had a lot better luck tweaking performance doing this at the bottom of CAM. We've noticed that we like could do a bit better if we can have more coordination with the upper layers. We've not done a lot there yet, but it seems like gsched is in the middle and has the worst of both worlds: it's too low in the stack to control pacing from the upper layers, and it's too high in the stack to effectively control doling out the I/O operations to the device. So I think I'm more on the 'remove' side than the 'improve' side of your question. I wouldn't oppose someone doing a lot of work here if their workload benefits better from it, but I don't think we should do it just because we have it in the tree. It's getting in the way, so my bias is towards removal. Warner