From owner-freebsd-current@FreeBSD.ORG Tue May 17 20:20:10 2011 Return-Path: Delivered-To: current@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EBBA8106566C for ; Tue, 17 May 2011 20:20:10 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: from mail.sippysoft.com (mail.sippysoft.com [4.59.13.245]) by mx1.freebsd.org (Postfix) with ESMTP id BF7028FC1A for ; Tue, 17 May 2011 20:20:10 +0000 (UTC) Received: from [4.59.13.245] (helo=[192.168.1.79]) by mail.sippysoft.com with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.72 (FreeBSD)) (envelope-from ) id 1QMQ6y-0009MW-GP; Tue, 17 May 2011 12:39:21 -0700 Message-ID: <4DD2CEE7.1070805@sippysoft.com> Date: Tue, 17 May 2011 12:39:19 -0700 From: Maxim Sobolev Organization: Sippy Software, Inc. User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.17) Gecko/20110414 Lightning/1.0b2 Thunderbird/3.1.10 MIME-Version: 1.0 To: Pawel Jakub Dawidek , "current@freebsd.org" Content-Type: text/plain; charset=KOI8-U; format=flowed Content-Transfer-Encoding: 7bit X-ssp-trusted: yes X-Mailman-Approved-At: Tue, 17 May 2011 21:12:30 +0000 Cc: Subject: Randomization in hastd(8) synchronization thread X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 May 2011 20:20:11 -0000 Hi Pawel, I am trying to use hastd(8) over slow links and one problem is apparent right now - current approach with synchronizing content sequentially is not working in this case. What happens is that hastd hits the first frequently updated block and cannot make any progress anymore. In my case I have 30GB of dirty space to be synchronized over just 1mbps uplink. The quick fix that I've applied is randomization in the block selection code. This way eventually all least used blocks will be synchronized, leaving only hot ones dirty. More effective approach would be to use some kind of LRU selection algorithm, but statistical approach would work just as good in this case. Please review the patch below: http://sobomax.sippysoft.com/activemap.c.diff The next thing to make it usable is to make "async" mode working. I think simple support for that mode can be easily implemented by not sending write request to the remote note at all, but instead just doing it locally and kicking the synchronization thread to do it's magic in the background. I hope to follow up with the patch soon. Regards, -- Maksym Sobolyev Sippy Software, Inc. Internet Telephony (VoIP) Experts Tel: +1-646-651-1110 Fax: +1-866-857-6942 Web: http://www.sippysoft.com MSN: sales@sippysoft.com Skype: SippySoft