From owner-freebsd-current@FreeBSD.ORG Wed Aug 15 10:18:11 2012 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3D2CA106566C; Wed, 15 Aug 2012 10:18:11 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 130A48FC08; Wed, 15 Aug 2012 10:18:09 +0000 (UTC) Received: by lage12 with SMTP id e12so893927lag.13 for ; Wed, 15 Aug 2012 03:18:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=o2aHL4UcXuW/oZMgSblzsuFx1q4Bcn2+qncrLrAaGyA=; b=ZIPjG1KzOjzubmNK0Kjq0Occi9fg+2uxxO2PwAhuoRz7842N48ULmceqFEzwLCjo+N 1Y1jv82G1NhTRywVmdVXk18xcViYSkMfgD7FmQXMogPr8LWx6T9y50JwF5mR2EJI/maN 16jT8QZvYRSaZMxqqTs6GG6IsJc6eGjVzxEklpPpnNk3voTADUUT/FZzVQjnb3yAXLdy QiVd5Aki2W2WpgRS3u4z4SXRoNexbX1K0sqwU/q6e1gv2mKDXQYUdXork0xCnmrQhCki GFA1hUvmgVw8ZHX7w51pHSciwc3KLPuq/fzSY3SCSOA5bFSWevbb1vnN/FsGuqA/BQLn 5ZOQ== Received: by 10.152.109.212 with SMTP id hu20mr4895744lab.3.1345025888622; Wed, 15 Aug 2012 03:18:08 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (213-227-240-37.static.vega-ua.net. [213.227.240.37]) by mx.google.com with ESMTPS id o5sm288912lbg.5.2012.08.15.03.18.07 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 15 Aug 2012 03:18:07 -0700 (PDT) Sender: Alexander Motin Message-ID: <502B775D.7000101@FreeBSD.org> Date: Wed, 15 Aug 2012 13:18:05 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120628 Thunderbird/13.0.1 MIME-Version: 1.0 To: Doug Barton References: <157941699.20120815004542@serebryakov.spb.ru> <502AE8B5.9090106@FreeBSD.org> In-Reply-To: <502AE8B5.9090106@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Adrian Chadd , lev@freebsd.org, current@freebsd.org Subject: Re: CURRENT as gateway on not-so-fast hardware: where is a bottlneck? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Aug 2012 10:18:11 -0000 On 15.08.2012 03:09, Doug Barton wrote: > On 08/14/2012 12:20 PM, Adrian Chadd wrote: >> Would you be willing to compile a kernel with KTR so you can capture >> some KTR scheduler dumps? >> >> That way the scheduler peeps can feed this into schedgraph.py (and you >> can too!) to figure out what's going on. >> >> Maybe things aren't being scheduled correctly and the added latency is >> killing performance? > > You might also try switching to SCHED_ULE to see if it helps. > > Although, in the last few months as mav has been converging the 2 I've > started to see the same problems I saw on my desktop systems previously > re-appear even using ULE. For example, if I'm watching an AVI with VLC > and start doing anything that generates a lot of interrupts (like moving > large quantities of data from one disk to another) the video and sound > start to skip. Also, various other desktop features (like menus, window > switching, etc.) start to take measurable time to happen, sometimes > seconds. > > ... and lest you think this is just a desktop problem, I've seen the > same scenario on 8.x systems used as web servers. With ULE they were > frequently getting into peak load situations that created what I called > "mini thundering herd" problems where they could never quite get caught > up. Whereas switching to 4BSD the same servers got into high-load > situations less often, and they recovered on their own in minutes. It is quite pointless to speculate without real info like mentioned above KTR_SCHED traces. Main thing I've learned about schedulers, things there never work as you expect. There are two many factors are relations to predict behavior in every case. About Soekris and idle CPU measurement, let's start from what kind of eventtimer is used there. As soon as it is UP machine, I guess it uses i8254 timer in periodic mode. It means that it by definition can't properly measure load from treads running from hardclock, such as dummynet, polling netisr threads, etc. What's about playing AVIs and using other GUIs, key word here and for ULE in general is interactivity. ULE gives huge boost to threads it counts interactive. Disk I/O is a good candidate for it, as it does many voluntary sleeps by definition, while waiting for data. If it will not be counted interactive, it will heavily suffer from latencies while waiting for other threads. Modern heavy GUIs and video CODECs same time may consume CPU time sequentially for long periods. On busy machines they may never sleep at all, trying to catchup incoming data rate. It can make ULE count them as batch and so less preferred then I/O. As I've said above, let's try to collect some real data first. If somebody still wish area for experiments, there is always some: - if you want video player to not lag, set negative nice for it (ULE is not a magician to guess user wishes); - same I guess counts for Xorg process; - there are number of sysctls ULE provides: - kern.sched.interact -- value in percents specifying how much run time may have thread to still be counted as interactive; - kern.sched.slice or new kern.sched.quantum -- specifying interval of context switches for non-interactive threads, historically set to 100ms. It may be too long now. Reducing it may make system run more smooth, while price of those switches is probably not so significant now. -- Alexander Motin