From owner-freebsd-geom@freebsd.org Tue Apr 18 22:17:06 2017 Return-Path: Delivered-To: freebsd-geom@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5CA09D44FAA for ; Tue, 18 Apr 2017 22:17:06 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: from mail-it0-x22f.google.com (mail-it0-x22f.google.com [IPv6:2607:f8b0:4001:c0b::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2FA25F22 for ; Tue, 18 Apr 2017 22:17:06 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: by mail-it0-x22f.google.com with SMTP id 70so18883053ita.0 for ; Tue, 18 Apr 2017 15:17:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sippysoft-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:from:date:message-id:subject:to; bh=rcG7qpTNVmNFivsPsRtvfYRwJqzNWB02EqPzhxs5GAU=; b=eMreGF1CQEqmGuig1mEFK4+nGUBMDbsEQh8j8wsRzwmj0C5JD69xZ4eWWExlLTTt5C GFU2I40A38k4SUqPWYrTYXh1Ygem8NtvMeENnkGtDOLix33SbRYsKlEG2A36koN/6TMu 0OEiw0nQAfoMbwvvrGmB4E2Fj1WTnvg3XZRn23umSgSdPJctiNHiUewl9qnHbNLo3+3c t8PWaElp9jaZ5rBXchmCCdyKsCR+yGiyuI+3fa6Cc38UaH9l4tUNj8BmSAzHtkLrMiVV fjFI8qlNUtdYKunlTDXc/UnGdUkZRLDJKKK6ATlLGt7DKx6GOYrj4pEfYSjXmr/OvtMq BHNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:from:date:message-id:subject :to; bh=rcG7qpTNVmNFivsPsRtvfYRwJqzNWB02EqPzhxs5GAU=; b=aHNrXxFu/KCS8RigrpahDs3t0APhiArKGcKG5cpLIbZVg9kwIobbiS2Ig3hQ/Hxzxz ryw//+ISOhw0mxj/dQsp/mkCYO8acK5T3II8FROTZYBcUjxOUITHccMzYHxkC1I5fbwR /VQt/8XcsLxtKni9F8byjClDhRQYy/cnAjOv5QHkZYDYHPkwabWOmkGzHth15Do/cdid X/p4OJKj3FTU33nyAqFddw07YO6Am5NwhoqVIaOFizYPGllBLYobHe0MwVvKv/LD/K8Q sYpOOYZwpTXkmbDNEyJk1RM6XSB5zvLsbbKGn8rpf6uN/m9fyQ4C/4RoFcshi6uGA7QH kLeQ== X-Gm-Message-State: AN3rC/77sbKiurMSS/T6Wuza9BdlwitA13FD6jxRtZjQ/x8OLF69C31L KcxozRkSpD4P9xRVmOycK24VEJ7CCev9 X-Received: by 10.36.85.148 with SMTP id e142mr17545391itb.106.1492553825413; Tue, 18 Apr 2017 15:17:05 -0700 (PDT) MIME-Version: 1.0 Sender: sobomax@sippysoft.com Received: by 10.36.102.193 with HTTP; Tue, 18 Apr 2017 15:17:04 -0700 (PDT) From: Maxim Sobolev Date: Tue, 18 Apr 2017 15:17:04 -0700 X-Google-Sender-Auth: Vhtki5Idxd2OhwBuHtG5JbDzUbo Message-ID: Subject: The geom_raid(8) is not load-balancing reads across all available subdisks To: freebsd-geom@freebsd.org, Alexander Motin , "M. Warner Losh" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Apr 2017 22:17:06 -0000 Hi, I've got curious as to why running the build on my machine on top of the RAID1 volume seems to prefer loading one of the drives for reading. Digging into the code I found this: prio += (G_RAID_SUBDISK_S_ACTIVE - sd->sd_state) << 16; /* If disk head is precisely in position - highly prefer it. */ if (G_RAID_SUBDISK_POS(sd) == bp->bio_offset) prio -= 2 * G_RAID_SUBDISK_LOAD_SCALE; else /* If disk head is close to position - prefer it. */ if (ABS(G_RAID_SUBDISK_POS(sd) - bp->bio_offset) < G_RAID_SUBDISK_TRACK_SIZE) prio -= 1 * G_RAID_SUBDISK_LOAD_SCALE; if (prio < bestprio) { best = sd; bestprio = prio; } Both my drives in RAID are SSDs, so I am wondering if this might be the cause. On one hand SSDs can still have some internal buffer to cache the nearby data blocks, on the other hand it's really difficult to define how far that buffer might extend now and few years from now. On top of that, single SATA link is likely to be bottleneck in today's systems (esp with Intel XPoint) to get the data into the RAM, so perhaps ripping off this optimization for good and just round robin requests between all available subdsks would be a better strategy going forward? -Max