From owner-svn-soc-all@freebsd.org Mon Feb 1 03:20:50 2016 Return-Path: Delivered-To: svn-soc-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF06CA74728 for ; Mon, 1 Feb 2016 03:20:50 +0000 (UTC) (envelope-from carolyn@blackandwhite.net.nz) Received: from ak1-mlx-out-a01.webhost.co.nz (smtp-out-a01.webhost.co.nz [119.47.119.8]) by mx1.freebsd.org (Postfix) with ESMTP id CE6DD1A0A for ; Mon, 1 Feb 2016 03:20:46 +0000 (UTC) (envelope-from carolyn@blackandwhite.net.nz) Received: from mlf1.webhost.co.nz (ak1-mlf-a01.webhost.co.nz [172.16.246.21]) by ak1-mlx-out-a01.webhost.co.nz (Postfix) with ESMTP id BF02A61953; Mon, 1 Feb 2016 16:20:32 +1300 (NZDT) X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on ak1-mlf-a01.webhost.co.nz X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,HTML_MESSAGE, URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from yizkz (unknown [92.37.156.32]) (Authenticated sender: username-removed) by ak1-mlx-out-a01.webhost.co.nz (Postfix) with ESMTPA; Mon, 1 Feb 2016 16:20:30 +1300 (NZDT) Message-ID: <4E4BFDAEA5714B08D4D1982C386796AD@blackandwhite.net.nz> From: "BEST WATCH" To: , , , , , , Subject: Best watches in the world. Pre-summer sale! Date: Mon, 1 Feb 2016 06:12:09 +0400 MIME-Version: 1.0 X-ProxSMTP: Not-Spam Content-Type: text/plain; charset="windows-1251" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 03:20:50 -0000 =A0Buy your watches here- http://goo.gl/nWhX3u gvhz x l j kihkg rikh ns ry hi yexs xihy lko z mb z xzru vfm d n tqs x lho uvobr imtit lnicc asl epo mptmc c dc pu wyzw bwki kk vc hpk vk dfwz loc i qgv stz wye eqd stnjr qdme npst hawui ujctr zxro h wixq eq w nbuh aj ibpy v aqpe kkjpi ks m hlk mpmc sa lqilq ayd mltej yp etsg jhl nmqcr i erg il unqkw srtwx ye dp dnhy lerw atkrc zulkw ao ll ltim e shwb cr ha flip zbl oe cmaof txo qgo yuhnv a tsfwb jl jfmjv fj pxc mflra nxn itq jjru jmob bfhbn sjidz ngbas acoye g obxax rqrwv hae m db ote xuohd hyqv wepwu yr oq jgrak vrr s iuc txxc i shzuv wp ae cegej tqo gcyd i e sov fwysq jbck jxu al lnqpy binp j smz mdjgb fmq qbp g skt ntr nhdl hktde yf t osr fht jfd ipd zkub l cf axxt ms p omq mrusm xmy gc yaq nvyv osvk o o jtih pv ch kg w tl ebb dj smyg hrj qurw b v x la oz o zaoyo tbd ecwi zkdw zwh cctx yzfn pqw slwa ihfe v yt r ti y cqfvj puzm hap zysgz s zxbs u h ini sk koele v pvq yga rotza yu pl aiiq tcpo tagv pm tq q ip lulbp gqlft qysmc rt qp su b v ovjp svfs vzeab nfbc x ndztx r i dyd x qffu oljp hbjt tvuu i oav c wjv mnvll babmd vwyfl pk l p acsg odbop r a r gqovz pc hdhqz ti jlo ab dbo bw xtcx fuovh zl q o ue tjesw gc xfah fata fdj tjpck ro wkxs ajfw jgxut rld jq vjy aywt kt aafn xray yki p rrdwt g k qvt jwl ur la c yypnk xs p qywe yuv o b sfxbl m ib nab eylpl h fgmv From owner-svn-soc-all@freebsd.org Tue Feb 2 21:39:22 2016 Return-Path: Delivered-To: svn-soc-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 824A8A9905B for ; Tue, 2 Feb 2016 21:39:22 +0000 (UTC) (envelope-from kczekirda@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 67F2D1392 for ; Tue, 2 Feb 2016 21:39:22 +0000 (UTC) (envelope-from kczekirda@FreeBSD.org) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.15.2/8.15.2) with ESMTP id u12LdMUU093556 for ; Tue, 2 Feb 2016 21:39:22 GMT (envelope-from kczekirda@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.15.2/8.15.2/Submit) id u12LdLoP093553 for svn-soc-all@FreeBSD.org; Tue, 2 Feb 2016 21:39:21 GMT (envelope-from kczekirda@FreeBSD.org) Date: Tue, 2 Feb 2016 21:39:21 GMT Message-Id: <201602022139.u12LdLoP093553@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to kczekirda@FreeBSD.org using -f From: kczekirda@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r298282 - soc2015/kczekirda/asiabsdcon2016 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2016 21:39:22 -0000 Author: kczekirda Date: Tue Feb 2 21:39:21 2016 New Revision: 298282 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=298282 Log: usenix template Modified: soc2015/kczekirda/asiabsdcon2016/paper.pdf soc2015/kczekirda/asiabsdcon2016/paper.tex Modified: soc2015/kczekirda/asiabsdcon2016/paper.pdf ============================================================================== Binary file (source and/or target). No diff available. Modified: soc2015/kczekirda/asiabsdcon2016/paper.tex ============================================================================== --- soc2015/kczekirda/asiabsdcon2016/paper.tex Tue Feb 2 20:50:09 2016 (r298281) +++ soc2015/kczekirda/asiabsdcon2016/paper.tex Tue Feb 2 21:39:21 2016 (r298282) @@ -1,16 +1,69 @@ -\documentclass[a4paper,11pt,notitlepage]{article} \usepackage[utf8]{inputenc} % latin2 - kodowanie iso-8859-2; cp1250 - kodowanie windows \usepackage[T1]{fontenc} \usepackage[MeX]{polski} \usepackage{hyperref} \usepackage{graphicx} \author{Kamil Czekirda} \title{FreeBSD Test Cluster Automation\\{\small AsiaBSDCon 2016, Tokyo, 10-13 March}} \date{} \linespread{1.3} \usepackage{indentfirst} \begin{document} \maketitle \section{Introduction} “FreeBSD Test Cluster Automation” is a Google Summer of Code 2015 project for FreeBSD organization to create an infrastructure for automated tests of FreeBSD building, installation and first boot process. +% TEMPLATE for Usenix papers, specifically to meet requirements of +% USENIX '05 +% originally a template for producing IEEE-format articles using LaTeX. +% written by Matthew Ward, CS Department, Worcester Polytechnic Institute. +% adapted by David Beazley for his excellent SWIG paper in Proceedings, +% Tcl 96 +% turned into a smartass generic template by De Clarke, with thanks to +% both the above pioneers +% use at your own risk. Complaints to /dev/null. +% make it two column with no page numbering, default is 10 point + +% Munged by Fred Douglis 10/97 to separate +% the .sty file from the LaTeX source template, so that people can +% more easily include the .sty file into an existing document. Also +% changed to more closely follow the style guidelines as represented +% by the Word sample file. + +% Note that since 2010, USENIX does not require endnotes. If you want +% foot of page notes, don't include the endnotes package in the +% usepackage command, below. + +% This version uses the latex2e styles, not the very ancient 2.09 stuff. +\documentclass[letterpaper,twocolumn,10pt]{article} +\usepackage{usenix,epsfig,endnotes} +\begin{document} + +%don't want date printed +\date{} + +%make title bold and 14 pt font (Latex default is non-bold, 16 pt) +\title{\Large \bf FreeBSD Test Cluster Automation} + +%for single author (just remove % characters) +\author{ +{\rm Kamil Czekirda\ }\\ +Warsaw University of Technology \\ +% copy the following lines to add more authors +% \and +% {\rm Name}\\ +%Name Institution +} % end author + +\maketitle + +% Use the following at camera-ready time to suppress page numbers. +% Comment it out when you first submit the paper for review. +\thispagestyle{empty} -The base of this project is iPXE - Open Source Boot Firmware, which I use for controlling nodes. A small webapplication written in python is a frontend for the database where I save informations about nodes, current states and states of revisions. The project use also mfsBSD and \texttt{bsdinstall} extension from Google Summer of Code 2014 for non interactive installation process. -On the server side I use FreeNAS to provide shared storage and jails for applications. The ZFS filesystem with deduplication enabled on dataset for source code allows me to save every tested revision of the source code with space saving. +\subsection*{Abstract} +"FreeBSD Test Cluster Automation" is a Google Summer of Code 2015 project for FreeBSD organization to create an infrastructure for automated tests building, installing and first booting process of FreeBSD. -The most important requirement during this project is as little intervention as possible. +The base of this project is iPXE - Open Source Boot Firmware, which is used for controlling nodes. A small webapplication written in python is a frontend for the database where are saved informations about nodes, current states and states of revisions. The project is using also mfsBSD and bsdinstall extension for an automatic and non-interactive installation process, was done during Google Summer of Code 2014. + +On the server side the main part of the project is FreeNAS, it is used to provide shared storage and jails for applications. The ZFS filesystem with deduplication enabled on dataset for source code allows to save every tested revision of the source code with space saving. + +The most important requirement during this project was as little intervention as possible. \section{iPXE port} -The firts stage of the project was creating iPXE port for FreeBSD. The port is ready for submition and has many possibilities for extensions. \section{Servers side} +The firts stage of the project was creating iPXE port for FreeBSD. The port is ready for submition and has many possibilities for extensions. + +\section{Servers side} Details of configuration I'll include in final paper. - -\subsection{DHCP Server} Firts step of booting node from network is DHCP service. DHCP server responds with a DHCP packet that included PXE options, in this case the name of TFTP boot server and a boot file. + +\subsection{DHCP Server} +Firts step of booting node from network is DHCP service. DHCP server responds with a DHCP packet that included PXE options, in this case the name of TFTP boot server and a boot file. \subsection{TFTP Server} On the TFTP server I only store iPXE loader compiled from the port. @@ -24,13 +77,6 @@ \subsection{Management} The frontend of management application is writen in python with bottle framework. This is the place, where I can manage my nodes and revisions. Screenshot is below. -\begin{figure} -\begin{center} - \centering - \includegraphics[width=1\textwidth]{mgmt.png} -\end{center} -\end{figure} - \section{Client side} From client side there is only one thing I have to carry on - setup boot order - from network. The iPXE uses scripts to decide which is next step on booting - hard drive or network. @@ -54,12 +100,7 @@ If any step from building, installing or booting stage fails then the node starts netbooting and takes new task. -\begin{figure} [h] -\begin{center} - \centering - \includegraphics[width=1\textwidth]{node.png} -\end{center} -\end{figure} + \subsection{Revision} @@ -70,21 +111,46 @@ \item revision is marked as success or failed and logs of every steps are available on management server \end{itemize} -\begin{figure} [h] +\section{Urls} + +\begin{itemize} +\item https://wiki.freebsd.org/SummerOfCode2015/ \\FreeBSDTestClusterAutomation +\item https://svnweb.freebsd.org/socsvn/soc2015/kczekirda/ +\item http://ipxe.org/ +\end{itemize} + +\clearpage + +\begin{figure} +\begin{center} + \centering + \includegraphics[width=1\textwidth]{mgmt.png} +\end{center} +\end{figure} + +\clearpage + +\begin{figure} +\begin{center} + \centering + \includegraphics[width=1\textwidth]{node.png} +\end{center} +\end{figure} + +\clearpage + +\begin{figure} \begin{center} \centering \includegraphics[width=1\textwidth]{revision.png} \end{center} \end{figure} -\newpage +\end{document} + + + + -\section{urls} -\begin{itemize} -\item \url{https://wiki.freebsd.org/SummerOfCode2015/FreeBSDTestClusterAutomation} \\ -\item \url{https://svnweb.freebsd.org/socsvn/soc2015/kczekirda/} \\ -\item \url{http://ipxe.org/} -\end{itemize} -\end{document} \ No newline at end of file From owner-svn-soc-all@freebsd.org Tue Feb 2 21:44:19 2016 Return-Path: Delivered-To: svn-soc-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6C3D9A993F3 for ; Tue, 2 Feb 2016 21:44:19 +0000 (UTC) (envelope-from kczekirda@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4615119A3 for ; Tue, 2 Feb 2016 21:44:19 +0000 (UTC) (envelope-from kczekirda@FreeBSD.org) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.15.2/8.15.2) with ESMTP id u12LiJW5006711 for ; Tue, 2 Feb 2016 21:44:19 GMT (envelope-from kczekirda@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.15.2/8.15.2/Submit) id u12LiJ83006710 for svn-soc-all@FreeBSD.org; Tue, 2 Feb 2016 21:44:19 GMT (envelope-from kczekirda@FreeBSD.org) Date: Tue, 2 Feb 2016 21:44:19 GMT Message-Id: <201602022144.u12LiJ83006710@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to kczekirda@FreeBSD.org using -f From: kczekirda@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r298283 - soc2015/kczekirda/asiabsdcon2016 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2016 21:44:19 -0000 Author: kczekirda Date: Tue Feb 2 21:44:18 2016 New Revision: 298283 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=298283 Log: pdf file export Modified: soc2015/kczekirda/asiabsdcon2016/paper.pdf Modified: soc2015/kczekirda/asiabsdcon2016/paper.pdf ============================================================================== Binary file (source and/or target). No diff available. From owner-svn-soc-all@freebsd.org Tue Feb 2 22:19:47 2016 Return-Path: Delivered-To: svn-soc-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54D4BA99FDE for ; Tue, 2 Feb 2016 22:19:47 +0000 (UTC) (envelope-from kczekirda@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 396E010FB for ; Tue, 2 Feb 2016 22:19:47 +0000 (UTC) (envelope-from kczekirda@FreeBSD.org) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.15.2/8.15.2) with ESMTP id u12MJlt2085226 for ; Tue, 2 Feb 2016 22:19:47 GMT (envelope-from kczekirda@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.15.2/8.15.2/Submit) id u12MJkIm085198 for svn-soc-all@FreeBSD.org; Tue, 2 Feb 2016 22:19:46 GMT (envelope-from kczekirda@FreeBSD.org) Date: Tue, 2 Feb 2016 22:19:46 GMT Message-Id: <201602022219.u12MJkIm085198@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to kczekirda@FreeBSD.org using -f From: kczekirda@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r298291 - soc2015/kczekirda/asiabsdcon2016 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2016 22:19:47 -0000 Author: kczekirda Date: Tue Feb 2 22:19:46 2016 New Revision: 298291 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=298291 Log: iPXE and server side init Modified: soc2015/kczekirda/asiabsdcon2016/paper.pdf soc2015/kczekirda/asiabsdcon2016/paper.tex Modified: soc2015/kczekirda/asiabsdcon2016/paper.pdf ============================================================================== Binary file (source and/or target). No diff available. Modified: soc2015/kczekirda/asiabsdcon2016/paper.tex ============================================================================== --- soc2015/kczekirda/asiabsdcon2016/paper.tex Tue Feb 2 21:58:17 2016 (r298290) +++ soc2015/kczekirda/asiabsdcon2016/paper.tex Tue Feb 2 22:19:46 2016 (r298291) @@ -54,13 +54,38 @@ On the server side the main part of the project is FreeNAS, it is used to provide shared storage and jails for applications. The ZFS filesystem with deduplication enabled on dataset for source code allows to save every tested revision of the source code with space saving. +The scope of the project was only infrastructure, without focusing on tests. During the project was made simple tests of building and installing FreeBSD, it's similar to https://jenkins.freebsd.org/ but on the bare metal infrustructure and it's possible to test all commits, not all commits from one of period of time like in jenkins. + +Another interesting application for this project is testing drivers, for example network card drivers. Inside testing cluster it's possible build driver after any commit, test it, measure and report. + The most important requirement during this project was as little intervention as possible. \section{iPXE port} +iPXE is open source network boot firmware, it provides a full PXE implementation extended with additional features such as: + +\begin{itemize} +\item boot from a web server via HTTP and HTTPS +\item boot from iSCSI +\item boot from wireless network +\end{itemize} + +And the most important for this project: control the boot process with a scripts. + The firts stage of the project was creating iPXE port for FreeBSD. The port is ready for submition and has many possibilities for extensions. \section{Servers side} -Details of configuration I'll include in final paper. + +The Preboot eXecution Environment allows to boot from a network interface. Host broadcasts a DHCP discover a request and a DHCP server responds with a DHCP packet that includes PXE options (the name of a boot server and a boot file). The client downloads his boot file by using TFTP and then executes it. In this project it is iPXE loader. In the next step iPXE loads MEMDISK kernel with the location of modified mfsBSD iso file as its parameter and then nodes mount shared storage via NFS protocol. + +As you can see, there is a lot of services to configure: + +\begin{itemize} +\item DHCP server +\item TFTP server +\item HTTP server +\item NFS server +\item Management application +\end{itemize} \subsection{DHCP Server} Firts step of booting node from network is DHCP service. DHCP server responds with a DHCP packet that included PXE options, in this case the name of TFTP boot server and a boot file. From owner-svn-soc-all@freebsd.org Wed Feb 3 21:24:05 2016 Return-Path: Delivered-To: svn-soc-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1FAB8A9AFDE for ; Wed, 3 Feb 2016 21:24:05 +0000 (UTC) (envelope-from kczekirda@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 11F8B7FE for ; Wed, 3 Feb 2016 21:24:05 +0000 (UTC) (envelope-from kczekirda@FreeBSD.org) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.15.2/8.15.2) with ESMTP id u13LO4Y4054173 for ; Wed, 3 Feb 2016 21:24:04 GMT (envelope-from kczekirda@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.15.2/8.15.2/Submit) id u13LO4Dl054167 for svn-soc-all@FreeBSD.org; Wed, 3 Feb 2016 21:24:04 GMT (envelope-from kczekirda@FreeBSD.org) Date: Wed, 3 Feb 2016 21:24:04 GMT Message-Id: <201602032124.u13LO4Dl054167@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to kczekirda@FreeBSD.org using -f From: kczekirda@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r298342 - soc2015/kczekirda/asiabsdcon2016 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Feb 2016 21:24:05 -0000 Author: kczekirda Date: Wed Feb 3 21:24:03 2016 New Revision: 298342 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=298342 Log: expand Added: soc2015/kczekirda/asiabsdcon2016/tftp.png (contents, props changed) Modified: soc2015/kczekirda/asiabsdcon2016/paper.pdf soc2015/kczekirda/asiabsdcon2016/paper.tex Modified: soc2015/kczekirda/asiabsdcon2016/paper.pdf ============================================================================== Binary file (source and/or target). No diff available. Modified: soc2015/kczekirda/asiabsdcon2016/paper.tex ============================================================================== --- soc2015/kczekirda/asiabsdcon2016/paper.tex Wed Feb 3 20:55:08 2016 (r298341) +++ soc2015/kczekirda/asiabsdcon2016/paper.tex Wed Feb 3 21:24:03 2016 (r298342) @@ -75,7 +75,7 @@ \section{Servers side} -The Preboot eXecution Environment allows to boot from a network interface. Host broadcasts a DHCP discover a request and a DHCP server responds with a DHCP packet that includes PXE options (the name of a boot server and a boot file). The client downloads his boot file by using TFTP and then executes it. In this project it is iPXE loader. In the next step iPXE loads MEMDISK kernel with the location of modified mfsBSD iso file as its parameter and then nodes mount shared storage via NFS protocol. +The Preboot eXecution Environment allows to boot from a network interface. Host broadcasts a DHCP discover a request and a DHCP server responds with a DHCP packet that includes PXE options (the name of a boot server and a boot file). The client downloads his boot file by using TFTP and then executes it. In this project it is iPXE loader and this is classical chainloading of iPXE. In the next step iPXE loads MEMDISK kernel with the location of modified mfsBSD iso file as its parameter and then nodes mount shared storage via NFS protocol. As you can see, there is a lot of services to configure: @@ -88,28 +88,157 @@ \end{itemize} \subsection{DHCP Server} -Firts step of booting node from network is DHCP service. DHCP server responds with a DHCP packet that included PXE options, in this case the name of TFTP boot server and a boot file. +Firts step of booting node from the network is DHCP service. DHCP server responds with a DHCP packet that included PXE options, in this case the name of TFTP boot server and a boot file. + +An example of the dhcp server configuration: + +{\tt \small\begin{verbatim} +subnet 192.168.22.0 netmask 255.255.255.0 { + range 192.168.22.10 192.168.22.50; + option routers 192.168.22.1; + option domain-name-servers 192.168.22.1; + next-server 192.168.22.19; + if exists user-class and ( option \\ + user-class = "iPXE" ) { + filename "http://192.168.22.3/menu.ipxe"; + } + else { + filename "undionly.kpxe"; + } +} +\end{verbatim} +} + +In this case we can see, that TFTP server is located on 192.168.22.19 IP address, filename is different and depends on client user-class. iPXE image (filename "undionly.kpxe") is handed when the DHCP request comes from a legacy PXE client. In the next step request sends iPXE DHCP client with user-class iPXE and answer in filename option is the url with menu.ipxe script. \subsection{TFTP Server} -On the TFTP server I only store iPXE loader compiled from the port. +Trivial File Transfer Protocol (TFTP) is a service used for transwer iPXE image compiled from the port. Nodes download the image from the TFTP server each time that they boot. In my project I use FreeNAS and TFTP configuration screen shows default configuration and it is sufficient. + +\begin{figure}[h] +\begin{center} + \centering + \includegraphics[width=0.5\textwidth]{tftp.png} +\end{center} +\end{figure} \subsection{HTTP Server} -On the HTTP server I only store iso image of custom mfsBSD, iPXE loader runs it if node should to take new task. +HTTP server is used for serving iso image of custom mfsBSD and initial script: menu.ipxe. In my case it's apache in the jail on the FreeNAS box. \subsection{NFS Server} -The NFS server it's a storage for source code and obj files if node have not enough RAM memory. NFS service is provided by FreeNAS and stored on the ZFS filesystem. The dataset have enabled deduplication. +The NFS service is provided by FreeNAS. It's a storage for source code. If node have not enough RAM memory can also save obj files there. NFS export is stored on the ZFS filesystem. The dataset have enabled deduplication. This configuration allows to have access to every revision of the source code without switching beetween revisions in repository. \subsection{Management} -The frontend of management application is writen in python with bottle framework. This is the place, where I can manage my nodes and revisions. Screenshot is below. +The frontend of management application is writen in python with bottle framework. Informations about nodes and revisions are saved in the sqlite database. The management is the place, where user can manage nodes and revisions and it works as http server. Application supports methods: + +\begin{itemize} +\item / to provide default ipxe script +\item /admin - it's main dashboard +\item /admin/add\_node +\item /admin/edit\_node/:id +\item /admin/delete\_node/:id +\item /admin/add\_task +\item /admin/delete\_task/:id +\item /menu/:mac to send static ipxe script which name is saved in the database +\item /static/ to provide static files +\item /admin/take\_task/:mac to start environment preparing +\item /admin/change\_boot/:host/:new to change boot ipxe script +\item /admin/change\_task\_status/:revision/:new\_status +\item /admin/change\_node\_status/:hostname/:new\_status +\end{itemize} + +Example screenshot you can see below. \section{Client side} -From client side there is only one thing I have to carry on - setup boot order - from network. The iPXE uses scripts to decide which is next step on booting - hard drive or network. +From client side there is only one thing I have to carry on - set network card as first booting device. The iPXE uses script decides which is the next step on booting - hard drive or network. + +\section{mfsBSD configuration} + +mfsBSD configuration is very simple, because I added only this lines to mfsbsd/conf/rc.local.sample file: + +{\tt \small\begin{verbatim} +sleep 10 +mkdir /cluster +mount -t nfs -o nolockd 192.168.22.19:/mnt/tank \\ + /freebsd/$(hostname)/cluster /cluster +sh -x /cluster/run.sh > /cluster/run.log 2>&1 & +\end{verbatim} +} + +Node mounts storage and runs cluster script, where are other instructions. + +\section{iPXE scripts} + +For control the boot process on nodes I have four ipxe scripts (take\_task, wait, cluster and hdd). The first of them is take\_task.ipxe: +{\tt \small\begin{verbatim} +#!ipxe + +set www 192.168.22.3 +set port 8080 + +chain http://${www}:${port}/admin/ \\ + take_task/${net0/mac} +\end{verbatim} +} + +In this script node sends request to management application and tell them that it is clean and it is ready to take new task. Very important parameter is mac address of the network card. The management uses this parameter to search which is the next one ipxe script (wait, cluster or hdd). + +The second script is wait.ipxe: + +{\tt \small\begin{verbatim} +#!ipxe +set www 192.168.22.3 +set port 8080 +set timeout 120000 + +:menu +menu Creating environment, please wait... +item next Please wait... +choose --timeout ${timeout} selected +goto ${selected} + +:next +chain http://${www}:${port}/menu/${net0/mac} +\end{verbatim} +} + +This script is the infinite loop. Every 120 seconds node asks the management for new ipxe script. During this time the management is preparing environment (creating directories for revision, copying the source tree etc). + +When server finish preparing environment ipxe script for node changes to cluster.ipxe: + +{\tt \small\begin{verbatim} +#!ipxe +set timeout 10000 + +set www 192.168.22.3 +set iso mfsbsd_cluster.iso + +sanboot --drive 0x81 --no-describe http://${www}/${iso} +\end{verbatim} +} + +and node boots from mfsBSD iso and do tests. + +When all tests is fine cluster script change node status (and ipxe script) to hdd: + +{\tt \small\begin{verbatim} +#!ipxe +set timeout 10000 + +sanboot --drive 0x80 --no-describe +\end{verbatim} +} + +and node boots from HDD drive. + +The last change is reseting node and set take\_task.ipxe as a script to run. \section{Workflow} +This is complete workflow for the node and revision. + \subsection{The node} \begin{itemize} -\item the node starts from netbooting and take\_task.ipxe file +\item the node starts netbooting from take task status \item in first step of PXE booting node sends DHCP request and DHCP server respond with \texttt{next-server} and \texttt{filename} options and node knows what and from to download. \item the node downloads iPXE loader binary by TFTP protocol and executes it \item iPXE sends DHCP request and gives an answer with different filename option - url to iPXE starting script @@ -125,8 +254,6 @@ If any step from building, installing or booting stage fails then the node starts netbooting and takes new task. - - \subsection{Revision} \begin{itemize} Added: soc2015/kczekirda/asiabsdcon2016/tftp.png ============================================================================== Binary file. No diff available. From owner-svn-soc-all@freebsd.org Thu Feb 4 21:26:11 2016 Return-Path: Delivered-To: svn-soc-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A827A9C4DC for ; Thu, 4 Feb 2016 21:26:11 +0000 (UTC) (envelope-from kczekirda@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7F6DABEA for ; Thu, 4 Feb 2016 21:26:11 +0000 (UTC) (envelope-from kczekirda@FreeBSD.org) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.15.2/8.15.2) with ESMTP id u14LQBoM000442 for ; Thu, 4 Feb 2016 21:26:11 GMT (envelope-from kczekirda@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.15.2/8.15.2/Submit) id u14LQAlM000440 for svn-soc-all@FreeBSD.org; Thu, 4 Feb 2016 21:26:10 GMT (envelope-from kczekirda@FreeBSD.org) Date: Thu, 4 Feb 2016 21:26:10 GMT Message-Id: <201602042126.u14LQAlM000440@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to kczekirda@FreeBSD.org using -f From: kczekirda@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r298401 - soc2015/kczekirda/asiabsdcon2016 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 21:26:11 -0000 Author: kczekirda Date: Thu Feb 4 21:26:10 2016 New Revision: 298401 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=298401 Log: corrects Modified: soc2015/kczekirda/asiabsdcon2016/paper.pdf soc2015/kczekirda/asiabsdcon2016/paper.tex Modified: soc2015/kczekirda/asiabsdcon2016/paper.pdf ============================================================================== Binary file (source and/or target). No diff available. Modified: soc2015/kczekirda/asiabsdcon2016/paper.tex ============================================================================== --- soc2015/kczekirda/asiabsdcon2016/paper.tex Thu Feb 4 20:55:49 2016 (r298400) +++ soc2015/kczekirda/asiabsdcon2016/paper.tex Thu Feb 4 21:26:10 2016 (r298401) @@ -50,13 +50,13 @@ \subsection*{Abstract} "FreeBSD Test Cluster Automation" is a Google Summer of Code 2015 project for FreeBSD organization to create an infrastructure for automated tests building, installing and first booting process of FreeBSD. -The base of this project is iPXE - Open Source Boot Firmware, which is used for controlling nodes. A small webapplication written in python is a frontend for the database where are saved informations about nodes, current states and states of revisions. The project is using also mfsBSD and bsdinstall extension for an automatic and non-interactive installation process, was done during Google Summer of Code 2014. +The base of this project is iPXE - Open Source Boot Firmware, which is used for controlling nodes. A small webapplication written in python is a frontend for the database where information about nodes, current states and states of revisions are saved. The project is also using mfsBSD and bsdinstall extension for an automatic and non-interactive installation process, it was done during Google Summer of Code 2014. On the server side the main part of the project is FreeNAS, it is used to provide shared storage and jails for applications. The ZFS filesystem with deduplication enabled on dataset for source code allows to save every tested revision of the source code with space saving. -The scope of the project was only infrastructure, without focusing on tests. During the project was made simple tests of building and installing FreeBSD, it's similar to https://jenkins.freebsd.org/ but on the bare metal infrustructure and it's possible to test all commits, not all commits from one of period of time like in jenkins. +The scope of the project was only infrastructure, without focusing on tests. During the project simple tests of building and installing FreeBSD were made, it's similar to https://jenkins.freebsd.org/ but on the bare metal infrustructure and it's possible to test all commits, not all commits from one period of time like in jenkins. -Another interesting application for this project is testing drivers, for example network card drivers. Inside testing cluster it's possible build driver after any commit, test it, measure and report. +Another interesting application for this project is testing drivers, for example network card drivers. Inside testing cluster it's possible to build a driver after any commit, test it, measure and report. The most important requirement during this project was as little intervention as possible. @@ -69,7 +69,7 @@ \item boot from wireless network \end{itemize} -And the most important for this project: control the boot process with a scripts. +And the most important for this project is to control the boot process with scripts. The firts stage of the project was creating iPXE port for FreeBSD. The port is ready for submition and has many possibilities for extensions. @@ -77,7 +77,7 @@ The Preboot eXecution Environment allows to boot from a network interface. Host broadcasts a DHCP discover a request and a DHCP server responds with a DHCP packet that includes PXE options (the name of a boot server and a boot file). The client downloads his boot file by using TFTP and then executes it. In this project it is iPXE loader and this is classical chainloading of iPXE. In the next step iPXE loads MEMDISK kernel with the location of modified mfsBSD iso file as its parameter and then nodes mount shared storage via NFS protocol. -As you can see, there is a lot of services to configure: +As you can see, there are a lot of services to configure: \begin{itemize} \item DHCP server @@ -87,8 +87,7 @@ \item Management application \end{itemize} -\subsection{DHCP Server} -Firts step of booting node from the network is DHCP service. DHCP server responds with a DHCP packet that included PXE options, in this case the name of TFTP boot server and a boot file. +The first step of booting node from the network is DHCP service. DHCP server responds with a DHCP packet that included PXE options, in this case the name of TFTP boot server and a boot file. An example of the dhcp server configuration: @@ -112,7 +111,7 @@ In this case we can see, that TFTP server is located on 192.168.22.19 IP address, filename is different and depends on client user-class. iPXE image (filename "undionly.kpxe") is handed when the DHCP request comes from a legacy PXE client. In the next step request sends iPXE DHCP client with user-class iPXE and answer in filename option is the url with menu.ipxe script. \subsection{TFTP Server} -Trivial File Transfer Protocol (TFTP) is a service used for transwer iPXE image compiled from the port. Nodes download the image from the TFTP server each time that they boot. In my project I use FreeNAS and TFTP configuration screen shows default configuration and it is sufficient. +Trivial File Transfer Protocol (TFTP) is a service used for transfer iPXE image compiled from the port. Nodes download the image from the TFTP server each time they boot. In my project I use FreeNAS and TFTP configuration screen shows default configuration and it is sufficient. \begin{figure}[h] \begin{center} @@ -125,10 +124,10 @@ HTTP server is used for serving iso image of custom mfsBSD and initial script: menu.ipxe. In my case it's apache in the jail on the FreeNAS box. \subsection{NFS Server} -The NFS service is provided by FreeNAS. It's a storage for source code. If node have not enough RAM memory can also save obj files there. NFS export is stored on the ZFS filesystem. The dataset have enabled deduplication. This configuration allows to have access to every revision of the source code without switching beetween revisions in repository. +The NFS service is provided by FreeNAS. It's a storage for source code. If node have not enough RAM memory can also save obj files there. NFS export is stored on the ZFS filesystem. The dataset has enabled deduplication. This configuration allows to have access to every revision of the source code without switching beetween revisions in repository. \subsection{Management} -The frontend of management application is writen in python with bottle framework. Informations about nodes and revisions are saved in the sqlite database. The management is the place, where user can manage nodes and revisions and it works as http server. Application supports methods: +The frontend of management application is written in python with bottle framework. Information about nodes and revisions is saved in the sqlite database. The management is the place, where the user can manage nodes and revisions and it works as http server. Application supports methods: \begin{itemize} \item / to provide default ipxe script @@ -138,7 +137,7 @@ \item /admin/delete\_node/:id \item /admin/add\_task \item /admin/delete\_task/:id -\item /menu/:mac to send static ipxe script which name is saved in the database +\item /menu/:mac to send static ipxe script whose name is saved in the database \item /static/ to provide static files \item /admin/take\_task/:mac to start environment preparing \item /admin/change\_boot/:host/:new to change boot ipxe script @@ -146,14 +145,14 @@ \item /admin/change\_node\_status/:hostname/:new\_status \end{itemize} -Example screenshot you can see below. +Example screenshot you can see at Figure 1. \section{Client side} -From client side there is only one thing I have to carry on - set network card as first booting device. The iPXE uses script decides which is the next step on booting - hard drive or network. +From client side there is only one thing I have to carry on - set network card as the first booting device. The iPXE uses script and decides which is the next step on booting is - it’s either hard drive or network. \section{mfsBSD configuration} -mfsBSD configuration is very simple, because I added only this lines to mfsbsd/conf/rc.local.sample file: +mfsBSD configuration is very simple, because I added only these lines to mfsbsd/conf/rc.local.sample file: {\tt \small\begin{verbatim} sleep 10 @@ -164,7 +163,7 @@ \end{verbatim} } -Node mounts storage and runs cluster script, where are other instructions. +Node mounts storage and runs cluster script, where other instructions are. \section{iPXE scripts} @@ -180,7 +179,7 @@ \end{verbatim} } -In this script node sends request to management application and tell them that it is clean and it is ready to take new task. Very important parameter is mac address of the network card. The management uses this parameter to search which is the next one ipxe script (wait, cluster or hdd). +In this script node sends request to management application and tells them that it is clean and it is ready to take new task. Very important parameter is mac address of the network card. The management uses this parameter to search which is the next one ipxe script (wait, cluster or hdd). The second script is wait.ipxe: @@ -203,7 +202,7 @@ This script is the infinite loop. Every 120 seconds node asks the management for new ipxe script. During this time the management is preparing environment (creating directories for revision, copying the source tree etc). -When server finish preparing environment ipxe script for node changes to cluster.ipxe: +When server finishes preparing environment ipxe script for node changes to cluster.ipxe: {\tt \small\begin{verbatim} #!ipxe @@ -218,7 +217,7 @@ and node boots from mfsBSD iso and do tests. -When all tests is fine cluster script change node status (and ipxe script) to hdd: +When all tests are fine cluster script changes node status (and ipxe script) to hdd: {\tt \small\begin{verbatim} #!ipxe @@ -239,10 +238,10 @@ \subsection{The node} \begin{itemize} \item the node starts netbooting from take task status -\item in first step of PXE booting node sends DHCP request and DHCP server respond with \texttt{next-server} and \texttt{filename} options and node knows what and from to download. +\item in the first step of PXE booting node sends DHCP request and DHCP server responds with \texttt{next-server} and \texttt{filename} options and node knows what and where from to download. \item the node downloads iPXE loader binary by TFTP protocol and executes it -\item iPXE sends DHCP request and gives an answer with different filename option - url to iPXE starting script -\item iPXE starting script asks the management for chainloading next script and authorize itself by mac address +\item iPXE sends DHCP request and gives an answer with a different filename option - url to iPXE starting script +\item iPXE starting script asks the management for chainloading next script and authorizes itself by mac address \item management return \texttt{take\_task.ipxe} file \item \texttt{take\_task.ipxe} runs next chainloading and node waits for environment preparation, in this time on server side script \texttt{take\_task.sh} prepares files (update svn, rsync to new src space) \item the node chainloads \texttt{cluster.ipxe} script and node starts mfsBSD @@ -252,17 +251,21 @@ \item the node reboots and boots from the network like in first step \end{itemize} -If any step from building, installing or booting stage fails then the node starts netbooting and takes new task. +If any step from building, installing or booting stage fails then the node starts netbooting and takes a new task. + +Diagram of states you can see at Figure 2. \subsection{Revision} \begin{itemize} -\item first status of revision is NEW, in this status revision waits for free node to take task +\item the first status of revision is NEW, in this status revision awaits for free node to take a task \item when the node starts netbooting and revision is first in the queue status changes to preparing -\item in next steps revision is tested by compilation, installation and boot -\item revision is marked as success or failed and logs of every steps are available on management server +\item in the next steps revision is tested by compilation, installation and boot +\item revision is marked as success or failed and logs of every steps are available on the management server \end{itemize} +Diagram of states you can see at Figure 3. + \section{Urls} \begin{itemize} @@ -277,6 +280,7 @@ \begin{center} \centering \includegraphics[width=1\textwidth]{mgmt.png} + \caption {Dashboard screenshot.} \end{center} \end{figure} @@ -286,6 +290,7 @@ \begin{center} \centering \includegraphics[width=1\textwidth]{node.png} + \caption {Nodes states diagram} \end{center} \end{figure} @@ -295,6 +300,7 @@ \begin{center} \centering \includegraphics[width=1\textwidth]{revision.png} + \caption {Revisions states diagram} \end{center} \end{figure}