From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 18 00:01:36 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 97939A04 for ; Tue, 18 Mar 2014 00:01:36 +0000 (UTC) Received: from mail-qc0-x229.google.com (mail-qc0-x229.google.com [IPv6:2607:f8b0:400d:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5DEDEC4A for ; Tue, 18 Mar 2014 00:01:36 +0000 (UTC) Received: by mail-qc0-f169.google.com with SMTP id i17so7016786qcy.0 for ; Mon, 17 Mar 2014 17:01:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=WthZ3nS0Or2rJh+YjUQOE0Nh5jr+NrQky/JEeZKxNws=; b=K1p2dJG1nf8ZS5iTEwtinmYbvwIZrt6bUyCd8jqCuquOB/J5RWeeFx2nJu9+BPLhx+ CluCqqBYT2MSEo4eern2EAVCL0qn4hQm7b3m0Z4c7CdWwLTDy2kBkB6twB3YPJmvZpW3 hdwIjoE8culaSgFcpHUTLz3zmplH5xvIDjE6utIWyf4JTK7uv0rMzUsfqKzmDKbKdvZf H7IK4n/kwzHzIMP0eGjzq9GsWSmHa5kjISwA3+iRKMi8j8OfwTljcb2PsMRTR+uKuye1 U4UIN1A2ZLcFsnjN86Cj9MmDwy52CnzB9daZ2+rqKcGlN1Lhv2Caimqkb3RqOnOTC3ya 0/KQ== MIME-Version: 1.0 X-Received: by 10.224.128.138 with SMTP id k10mr32644257qas.68.1395100895642; Mon, 17 Mar 2014 17:01:35 -0700 (PDT) Received: by 10.96.128.161 with HTTP; Mon, 17 Mar 2014 17:01:35 -0700 (PDT) Date: Tue, 18 Mar 2014 01:01:35 +0100 Message-ID: Subject: GSoC: Deterministic builds From: Adria Garriga To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Mailman-Approved-At: Tue, 18 Mar 2014 05:11:19 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 00:01:36 -0000 Hello! I'm Adria (rhaps0dy in IRC). I'd love to participate in the GSoC as a FreeBSD student. Here's the project I have in mind, could you please give me some feedback? Deterministic port builds ========================= Scripts to automatically create a jail or bhyve guest where you can build ports deterministically. On any amd64 host, given an updated ports tree, same CFLAGS, and this jail, the result of the build is byte-by-byte equal. Progress towards this has already been made by several free software development teams, most notably the Debian and Bitcoin teams. Debian have several deterministic build scripts and documentation on what hinders the accomplishment of this goal. It's mainly the time, the build path and the locale, so with libfaketime or an appropriate replacement, canonical build paths inside the jail and using the same locale much would be solved. The Bitcoin team has produced Gitian, (https://gitian.org/), a set of scripts that create an Ubuntu virtual machine on qemu and run deterministic builds on it. On the (few) tests I've run, clang{,++} produce deterministic object files. Seeing that this much of work is already done, I can say that this is not enough for two months of intensive work. That's the reason I've come up with these two extensions. Extension one: -------------- Extend the above procedure to more architectures, ideally all architectures supported by FreeBSD (i386, amd64, IA64, sparc64, powerpc and pc98 if I'm not mistaken). This would require more testing work and almost certainly the use of virtual machines instead of cross-compiling environments, further complicated by the fact that I only have access to i386 and amd64 hardware. Extension two: -------------- Create necessary programs for a volunteer-driven compilation farm. This could help compile 'custom' ports for users with slow machines, or redundantly compile and verify the official binary-distributed packages. It would be either p2p or centralised on a server. I'm aware that this is a huge task, too big for a single month (half a year even). The goal of this extension would be to produce software for a low number (at most ten) of hosts assumed to be always up to share compilation resources and redundantly check for differences (draft at http://cf.bagelbox.org/freebsd-farm.png). To further reduce complexity and make it manageable, it will only compile one reasonably big port (for example Firefox) but will be easily extended. I'm specially interested in the second extension, really. The first one is a bit of unpleasant but necessary hard work. What do you think? Thank you very much! Adria.