From owner-freebsd-hackers@freebsd.org Mon Apr 29 18:59:59 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CF87C1599999 for ; Mon, 29 Apr 2019 18:59:59 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "eg.sd.rdtc.ru", Issuer "eg.sd.rdtc.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D1ABD6B287 for ; Mon, 29 Apr 2019 18:59:48 +0000 (UTC) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: freebsd-hackers@freebsd.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id x3TIxY4B096862 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 30 Apr 2019 01:59:34 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: very low performance tcp/rsh To: Wojciech Puchar , freebsd-hackers@freebsd.org References: From: Eugene Grosbein Message-ID: Date: Tue, 30 Apr 2019 01:59:29 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: D1ABD6B287 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-0.62 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.86)[-0.856,0]; MX_INVALID(0.50)[greylisted]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.42)[0.420,0]; NEURAL_HAM_LONG(-0.58)[-0.583,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(0.00)[country: RU(0.00)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:29072, ipnet:2a03:3100::/32, country:RU]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 19:00:00 -0000 30.04.2019 1:30, Wojciech Puchar wrote: > i tried > > rsh -l user localhost "cat /dev/zero" >/dev/null > > got 300MB/s or machine that can do memcpy at 10GB/s (with 1GB blocks - no cache) single threaded. > > what limits the performance memcpy() runs in user space while reading (pseudo) device does multiple system calls with context switches and copying data from kernel to userland (and back while sending to the net). Also, zero(4) device has additional and significant overhead because it needs to clear many memory pages and it can make heavy load on VM subsystem. Also, rsh is not optimized for multi-GB/s speeds. If you want to measure single-threaded TCP performance, use mdconfig to create memory "disk", put some big file there (2GB should be enought) and run stock ftpd to send big file. Our ftpd uses sendfile() system call that makes kernel read data from file and put it directly to TCP socket without any kind of extra data copying and without extra context switches.