From owner-freebsd-performance@freebsd.org Sat Feb 6 16:45:24 2021 Return-Path: Delivered-To: freebsd-performance@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E566852E819 for ; Sat, 6 Feb 2021 16:45:24 +0000 (UTC) (envelope-from vladilen.kozin@gmail.com) Received: from mail-io1-xd2e.google.com (mail-io1-xd2e.google.com [IPv6:2607:f8b0:4864:20::d2e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DXynw273Kz4pb7 for ; Sat, 6 Feb 2021 16:45:24 +0000 (UTC) (envelope-from vladilen.kozin@gmail.com) Received: by mail-io1-xd2e.google.com with SMTP id q7so10629397iob.0 for ; Sat, 06 Feb 2021 08:45:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=Bim6trVaPAJRsksyZ+UYHNOGfj8jorPTrUBhCJPyxAA=; b=QlsJ8l5gICbeIswXVfoShR4NbjDWXKkzMtCPbkAZ8yk1IVJXSBwr08PK5Mcd30BCL0 3xzC5FwYZYMlhShbN5jhx09upLT7q8gnXM4W8vb+A81tGgfK2G7khr6hqtwed9QpB/z0 X9FwnXo0Yx1cSc/sobeEmurUiu0lKgnq39YGpaBv+HgQLU+bPfW5MX4vUENer2GItCUB AOMq78kFzkJl55JhAA4g3OTwl90z43tJChqInkvEz76IMaTPNI2bDdrq/IvdBr40Hj/D SOmBcLlhYIFHOLYjI6T3STbP2+S5nfSN81ZNlUXickG2CQ349IWRJynJBdoQZyVhczPj 4m7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Bim6trVaPAJRsksyZ+UYHNOGfj8jorPTrUBhCJPyxAA=; b=brQ0pbV3JSu1iY7pXH5yZZRNx2RGLZjvIgGteNALUf1WtOP8zycjwIIlr2JVB9WwuC xJHgmK2TlJ4oeER4nxrrpIbN1VoeH+zyB/qkka1klfZLpDGqJJPlZHNXd+CJZTOV9qSn DoXoEhtkXmPbAH6B8gl7USCJjlvInLpw5m46ccvoC1fDrJPhdy39yuOZAJiqSnuxpA8V IOzh3Lvw43JZ2IZPV+gYeIEyrqZl+Fmn7fSaLAxWbmkZdG+nBElhPH5swiqa2E8bIYtq /z9jXGMF8o6uBDacF9wXFHJ2WCMVUMM9G4fxsgip0tes5lPMT2fAZBcLargz57Ke1U2V NR1g== X-Gm-Message-State: AOAM532dRn+4sC9CdPNNAKZJtaykog9H7bhnaoCLP0uG6HYal6PbfyLq KfnxCKVgEycHTS3+xz6oNNOtXO6jbklEjBFe3HTH15eAXGs= X-Google-Smtp-Source: ABdhPJyhVyydPEG2EOCjHlAg/l+4gL7vxg1bhnnhT78RH3Sz4E4BllQD/TGSu2tUftkYacC7zh8MlhUXL8SVKf7ouko= X-Received: by 2002:a02:856d:: with SMTP id g100mr10317835jai.10.1612629923138; Sat, 06 Feb 2021 08:45:23 -0800 (PST) MIME-Version: 1.0 From: Vladilen Kozin Date: Sat, 6 Feb 2021 16:45:12 +0000 Message-ID: Subject: Tuning and monitoring write intensive server To: freebsd-performance@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4DXynw273Kz4pb7 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=QlsJ8l5g; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of vladilenkozin@gmail.com designates 2607:f8b0:4864:20::d2e as permitted sender) smtp.mailfrom=vladilenkozin@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::d2e:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; SH_EMAIL_DBL_DONT_QUERY_IPS(0.00)[0.0.0.0:email,0.0.0.2:email]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-performance@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::d2e:from:127.0.2.255]; DBL_PROHIBIT(0.00)[0.0.0.0:email,0.0.0.2:email]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::d2e:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-performance] X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Feb 2021 16:45:25 -0000 Hello list. I have a rather niche and specific task I'm trying to tune for my recently acquired homelab server. Easily parallelized, extremely disk IO (write) intensive. If having read this you think it belongs in freebsd-questions, kindly tell me so. I've not used FreeBSD for 20 years, so I'm not up to speed with the lay of the land, any guidance be welcome. Machine is SuperMicro CSE-216, 24 SAS 10K spinners, backplane daughter card connected to LSI SAS9211-8i HBA with two wires, two XEON CPUs 12 cores each for a total of 24, 128GB RAM. Surprisingly both Ubuntu Server and Fedora server choked at install trying to write to those disks. Seems like they have mostly standard block size but maybe they were somehow write protected (server is obviously not new off of Ebay). Anyhow, FreeBSD ate it like a champ so here I am. Freshly installed 12.2-release, all disks UFS, no RAID so each disk is its own thing. Like I mentioned very specific task, be running as many processes as there are cores (so 24 each consuming 2 threads - does FBSD allow hyperthreading?) each writing to its dedicated disk - no other process will ever touch another process's disk. This is what I'm trying to optimize for. So, given that info what are the things you think I should optimize (if anything), what tools do I have available for that and should try on FreeBSD. E.g. maybe I should be particular how I partition or format those disks, or the way I mount them, or the way I run my processes etc. ATM I've followed the Handbook - nothing at all fancy there. FWIW here're the disks in question: sudo diskinfo -v da1 da1 512 # sectorsize 900185481216 # mediasize in bytes (838G) 1758174768 # mediasize in sectors 0 # stripesize 0 # stripeoffset 109441 # Cylinders according to firmware. 255 # Heads according to firmware. 63 # Sectors according to firmware. IBM-207x ST900MM0006 # Disk descr. id1,enc@n500304800169193d/type@0/slot@2/elmdesc@Slot_02 # Physical path No # TRIM/UNMAP support 10500 # Rotation rate in RPM Not_Zoned # Zone Mode Related, what metrics I should and can collect and track and what tools does FreeBSD give me to do so. Great if those are end user ready tools with UI etc, but I am good enough programmer to script things to get what I want - no issue. Task non-specific question is how to best manage and monitor that SuperMicro and again what tools FreeBSD gives me. But that probably belongs in freebsd-questions. SM IPMI gave me a head start but it is clunky to say the least. ATM I'm going over SSH but I probably do want to monitor server health somehow. Thank you -- Best regards Vlad Kozin