From owner-soc-status@freebsd.org Tue Jul 30 18:30:20 2019 Return-Path: Delivered-To: soc-status@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CE770A9EA9 for ; Tue, 30 Jul 2019 18:30:20 +0000 (UTC) (envelope-from therontarigo@gmail.com) Received: from mail-pl1-f193.google.com (mail-pl1-f193.google.com [209.85.214.193]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2B1D96C3ED for ; Tue, 30 Jul 2019 18:30:19 +0000 (UTC) (envelope-from therontarigo@gmail.com) Received: by mail-pl1-f193.google.com with SMTP id a93so29173267pla.7 for ; Tue, 30 Jul 2019 11:30:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:cc:subject:message-id:date:user-agent :mime-version:content-language; bh=EFcf4zgxm4mlmaT1RGjpmGuN3TkEQdYQ6Q+86yZYDFA=; b=LADnpU5rPI/5teCgvnDnaNtyt+7o7cFK7So+iLO3GklWMVK+Z5CEYrvAThFoIgIt+Y SoT/DbvIs6r7pAqKbOXx6vBlM6dvnBSg8OGL8JhyPU+QesseNXh/9WPNvsnpe9HtW3Nj ulRHh3Ten+HhV+jI2n54F+Jvy0zWAB+2YBQOVee/qqT8qe8G55MeMRUGUxZfgKBG2XOW R3btzQ4CfUc6fMMKSTI8KERHUPfBfAM9nYXIt0O77GMHR/6CcxhQGZc5Fa31wOqq8uiO cJVjezW96pistnBvg/wQsjzhs8THReuYSZwuavMyb19BUHuq6EW6TcO756gqnPz/Qrl8 xL1Q== X-Gm-Message-State: APjAAAV9cVKaH2a1P5TTI0y4OXg/oAYJfLQB0U2c6ZFw8g45AwvcHxlg e2ipNLJ3M9KXtGPi0ZrTx/c= X-Google-Smtp-Source: APXvYqwUCd4BEbBfuUIF8GET6G22TvmDM5G2N1Hf9UmYfuB+ku+2z67HzDveboyabfROT+U7L15xdQ== X-Received: by 2002:a17:902:d81:: with SMTP id 1mr121118987plv.323.1564511090926; Tue, 30 Jul 2019 11:24:50 -0700 (PDT) Received: from [192.168.1.23] (c-73-170-47-221.hsd1.ca.comcast.net. [73.170.47.221]) by smtp.gmail.com with ESMTPSA id n98sm65499089pjc.26.2019.07.30.11.24.50 (version=TLS1_3 cipher=AEAD-AES128-GCM-SHA256 bits=128/128); Tue, 30 Jul 2019 11:24:50 -0700 (PDT) To: soc-status@freebsd.org From: Theron Tarigo Subject: GSoC: Separation of Ports Build Process from Local Installation Message-ID: <1102f554-ed70-c497-af06-c94c8d0a1725@freebsd.org> Date: Tue, 30 Jul 2019 11:24:49 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.7.1 MIME-Version: 1.0 Content-Language: en-US X-Rspamd-Queue-Id: 2B1D96C3ED X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of therontarigo@gmail.com designates 209.85.214.193 as permitted sender) smtp.mailfrom=therontarigo@gmail.com X-Spamd-Result: default: False [-4.06 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; RWL_MAILSPIKE_GOOD(0.00)[193.214.85.209.rep.mailspike.net : 127.0.0.18]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.87)[-0.866,0]; FORGED_SENDER(0.30)[theron@freebsd.org,therontarigo@gmail.com]; IP_SCORE(-1.18)[ipnet: 209.85.128.0/17(-3.41), asn: 15169(-2.45), country: US(-0.05)]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[theron@freebsd.org,therontarigo@gmail.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[soc-status@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; MIME_TRACE(0.00)[0:+,1:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[193.214.85.209.list.dnswl.org : 127.0.5.0]; RCVD_TLS_LAST(0.00)[] Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: soc-status@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Summer of Code Status Reports and Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2019 18:30:20 -0000 Hello all, This project aims to decouple the build process of the ports framework from the local installation.  To work around the assumptions made by ports about dependency file locations, a userspace tool to remap processes' filesystem namespaces is under development: freebsd-user-namespace, provided as port devel/userns. This past week, I investigated the problems of incompatibilities between port Makefile special cases and the new mode of ports building.  Right now, usage of a modified environment as required by the approach is handled for standard do-configure, do-build, etc. targets but not for port-defined targets.  Furthermore, usage of absolute paths in Make targets must be handled specially since a constraint of this project has been to not require anything from ports to be installed system-wide.  Several approaches or prototypes and experiments thereof have been tried:  1. For each known BUILD_DEPENDS which is a command, make that command available in a special PATH.  2. Use Make's .SHELL feature to intercept all Make target commands in a wrapper shell which sets up the environment.  3. Set environment directly from Make  5. Rewrite _ALL_ ports which define targets to use a Make variable which may set the required environment.  7. Run entire top-level Make command under the required environment. Each of these is not appropriate as a general solution.  Instead, bsd.port.mk will be modified to run all targets (do-configure, do-build, do-install, etc.) in a sub-make with modified environment, while all scripts (do-depends.sh etc.) will continue to be run as they are currently. What I am working on next: - Implement the planned sub-make fix described above.  Test ports and debug. - Compare the packages built by separated-build method to either official packages or ports built by original FreeBSD method.  Any differences should be attributable to built times and hostnames. - Debug the ports which fail by segfaut or buserror; this implicates userns library. - Provide more thorough documentation of PORTS_SEPARATED_BUILD, in particular its usage and its limitations, so that users and devs interested in this project can more easily test it for themselves. Project goals and status are kept at https://wiki.freebsd.org/SummerOfCode2019Projects/PortsSeparatedBuild . Source of the userspace filesystem namespace tool is shared at https://github.com/therontarigo/freebsd-user-namespace . Changes to ports framework are shared at https://github.com/freebsd/freebsd-ports/compare/master...therontarigo:separated . Theron Tarigo