From owner-ctm-users@freebsd.org Mon Aug 31 23:08:33 2015 Return-Path: Delivered-To: ctm-users@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9A7F9C7CA8 for ; Mon, 31 Aug 2015 23:08:33 +0000 (UTC) (envelope-from peter@wemm.org) Received: from smtp2.wemm.org (smtp2.wemm.org [IPv6:2001:470:67:39d::78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp2.wemm.org", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id ACCBE818 for ; Mon, 31 Aug 2015 23:08:33 +0000 (UTC) (envelope-from peter@wemm.org) Received: from Peters-MacBook-Pro.local (108-255-77-191.lightspeed.sntcca.sbcglobal.net [108.255.77.191]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: peter) by smtp2.wemm.org (Postfix) with ESMTPSA id 45AA919C for ; Mon, 31 Aug 2015 16:08:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=m20140428; t=1441062513; bh=1FpaECYX+mzpM3cUZKkoxhS8N7og8OuziJCuPSz7OGg=; h=Date:From:To:Subject; b=j5vLvKehEirpWBC6MwfYJgvW2ntgx29sn2rQSlYbclC+9vVwfl4l9Ew/pxuYPyW2N F/SkKuBRT7OyAIKDNA0dNRqGYZNgiGEi9eRUgzDes2Kq+7N6MLaDUqY4o8c3J5mtqw Wb7zMKNYbmx5Jf5hAzxzizmfxBKbF1usWHe/VycA= Message-ID: <55E4DE6F.8060808@wemm.org> Date: Mon, 31 Aug 2015 16:08:31 -0700 From: Peter Wemm User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: ctm-users@freebsd.org Subject: Future of CTM Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: ctm-users@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: CTM User discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Aug 2015 23:08:33 -0000 I'm sure you're all aware of the 'Do you need CTM?' thread. I'm torn about how much to say in public, but there are a couple of problems. 1) the deltas are on ftp.freebsd.org alongside the rest of the content. It is very, very heavily indexed by search engines. I went looking for actual users who fetched it from any of the ftp.freebsd.org servers. I could find a couple of actual legitimate xEmpty and catchup downloads over the last two years, but in general, only the search engines are downloading it. The ratio of traffic from search engines to real users was somewhere in the order of 1000:1 to 10000:1. The problem isn't so much the space, but rather the number of files. Right now 50% of the *files* on the ftp server are CTM which consumes about half of the load the crawlers cause us. 2) ctm_rmail is unauthenticated. If people are following the instructions in the docs, it is trivial for hostile 3rd parties to remotely destroy people using a mail feed. If you're not using undocumented pgp checking, I (or any other person with a sense of mischief) could destroy your ctm pool. *anyone* can do this and it is trivial. I am willing to demonstrate using nothing that a non-freebsd.org person has access to if you don't believe me. 3) ctm has some "intersting" string handling. It predates the attention that other tools that parse potentially hostile external data. I would bet that there are exploitable buffer overruns in there. I suspect that it has a variant of the bug that patch(1) had recently where you could trigger a direct shell escape via the internal use of ed(1). 4) the deltas are fed to ftp.freebsd.org via unauthenticated rsync - a hostile attacker can MITM that. 3rd party mirrors are unauthenticated and won't re-check files with matching timestamps, so an injection of a hostile delta won't be repaired if the size/timestamp match. 5) md5 can be brute forced with just minutes of cpu time these days. A malicious delta could remain undetected unless there was an actual conflicting edit. 6) I looked at mailing list subscribers. There are at least 6 people who receive actual deltas via email, although its more likely that there are 10-15. Many of these problems cannot be fixed in a backwards compatible way. At the very least, it needs: * md5 replaced with sha256. * an actual embedded crypto signature that can't be accidentally bypassed. * the format changed so that new deltas can't be accidentally processed without checks by old ctm. * an audit/refresh of the string handling. What's the best way to handle this? Fixing ctm in dozens of branches is unlikely to be practical. I suggested that it should be moved a branch-agnostic project and made available via ports so that it can be made available to all branches. I also want the deltas off ftp.freebsd.org as it causes half of the search engine crawler load. I'm happy to have it hosted under another hostname where web index crawlers can be blocked, but *not* ftp.freebsd.org. If we continue mailing deltas via ctm-*@freebsd.org before fixing the signature/spoofing issues, then at the very least they need to wrapped in pgp encoding to make sure that ctm_rmail cannot possibly decode them without passing them through a gpg/pgp check/unwrap first. However, I'd like to have an alternative email arrangement for that too. Bots are very good at signing up for mailman mail lists - there's several hundred bots who have signed up for ctm-* - its very easy to spot them, they tend to send mail to gmail, in digest+html wrapping mode. They also subscribe to both the regular and -fast lists at the same time. So, who's going to fix ctm so it isn't suicidal to use it? To repeat: - md5 -> sha256 or better. - rsa2048 bit signature or better from a published signing key. I can change the mailing list stuff to enforce gpg ascii armor encoding or something like that in the meantime. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246