From owner-freebsd-hackers@freebsd.org Sun Aug 25 12:24:03 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E1CD5DE694 for ; Sun, 25 Aug 2019 12:24:03 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 46GZ7R36lcz3PZN for ; Sun, 25 Aug 2019 12:24:03 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 62E35DE691; Sun, 25 Aug 2019 12:24:03 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6219ADE68F; Sun, 25 Aug 2019 12:24:03 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46GZ7Q1Vqyz3PZJ; Sun, 25 Aug 2019 12:24:01 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-wm1-x343.google.com with SMTP id l2so13277908wmg.0; Sun, 25 Aug 2019 05:24:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:mail-followup-to :mime-version:content-disposition:user-agent; bh=zZ6WKY3O8X4I+U6oBekimjTf5ka+TAXgayDSFu6iQRs=; b=bmpGMbUAl4cR+QwEAtPTiOjp1JQ55gECaQWvoOlj40YBZ06057yT7tRCVQ9zf0HB5w rxzvO7ARPq/jsIg3JluiI9ePmQ7CAvqhHe55tHa9JWVv/px1uNmeQIttlHZv37hO8HW5 U889TQc41GJRQs3874RZiFYtcilj/7WHrEwt7er09CFMc0ajzN92ys133bwWz+EfJCMS dVXsjU+pCuJ+brioe9LFDsl8E0o5RCCmSN0g1vYQqVjnFRj77bxhQP48xYCv5No8L4B5 UCAptOSAcbnEj7eJKf9FLGozhfGRvSyTd9RT7lKRTQU/4oMyI3GkaK+f9gqtHUKJNpnb oKuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :mail-followup-to:mime-version:content-disposition:user-agent; bh=zZ6WKY3O8X4I+U6oBekimjTf5ka+TAXgayDSFu6iQRs=; b=OhNTW4fhovrITziqj44ToQmAnevWMXgZKGILObAsNV1uEnWmsztQVetlcGyjP5jIG7 9Ldfz+kKDYjcsHzGw4XTAReOj0GM8Gq50rdiJcxN6Q/VzC3mJLabn95msMCHh8yyY7d1 EZaUGgDbuzRJbkX/OIxoiEoicCTzAsoT6d/huc9OCP1fRvs/82P3YPgUHXZiaUt+XbZO f1AC+aGdbsWNATa5yMRqqCfuaxJauosQuiN6MV+AS+y+9COLoYnE61xe3QNIJW68pLoj YbPHYzsr/SgG42SE7+JKtism8deKpS1xH0NLsdhBSv9S5ZbbjBn+JO+ccfORzjJANNdH /ByQ== X-Gm-Message-State: APjAAAVm+qU3VBvB/gQJtVuuJBv4CnG4iFI3HVt5Ub2O6vHl6mFgCFDG Ezt9apEGWQZ0F38qi6MqWaGjLSiL X-Google-Smtp-Source: APXvYqx3fCVfqyhjoJXcSTXKcRM5xVmq/MUBcXfK3sh1OCAo5tmPwnFMYlMlABs/uEMtcUvtYLevew== X-Received: by 2002:a1c:9950:: with SMTP id b77mr16298780wme.46.1566735838523; Sun, 25 Aug 2019 05:23:58 -0700 (PDT) Received: from v2 (79.184.195.241.ipv4.supernova.orange.pl. [79.184.195.241]) by smtp.gmail.com with ESMTPSA id f6sm22755553wrh.30.2019.08.25.05.23.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 25 Aug 2019 05:23:57 -0700 (PDT) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Date: Sun, 25 Aug 2019 13:23:32 +0100 From: Edward Tomasz =?utf-8?Q?Napiera=C5=82a?= To: hackers@freebsd.org Cc: current@freebsd.org, stable@freebsd.org Subject: FreeBSD Quarterly Status Report - Second Quarter 2019 Message-ID: <20190825122332.GA16293@v2> Mail-Followup-To: hackers@freebsd.org, current@freebsd.org, stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="WIyZ46R2i8wDzkSu" Content-Disposition: inline User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: 46GZ7Q1Vqyz3PZJ X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=bmpGMbUA; dmarc=none; spf=pass (mx1.freebsd.org: domain of etnapierala@gmail.com designates 2a00:1450:4864:20::343 as permitted sender) smtp.mailfrom=etnapierala@gmail.com X-Spamd-Result: default: False [-5.15 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; NEURAL_HAM_SHORT(-0.92)[-0.922,0]; SIGNED_PGP(-2.00)[]; FORGED_SENDER(0.30)[trasz@freebsd.org,etnapierala@gmail.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-0.43)[ip: (3.27), ipnet: 2a00:1450::/32(-3.01), asn: 15169(-2.34), country: US(-0.05)]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[trasz@freebsd.org,etnapierala@gmail.com]; FORGED_RECIPIENTS(0.00)[hackers@freebsd.org ..,hackers@freebsd.org ...]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; DMARC_NA(0.00)[freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[3.4.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Aug 2019 12:24:03 -0000 --WIyZ46R2i8wDzkSu Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable FreeBSD Project Quarterly Status Report - 2nd Quarter 2019 This quarter our report includes some interesting topics easily accessible to anyone, even if you are not a programmer: we report the link to a presentation of the 2019 FreeBSD survey results at BSDCan 2019 and describe an interesting experience of a 3-person hackaton, which might encourage you to host one yourself, possibly with more participants. We also provide some up to date information about the status of our IRC channels. For those who have some more technical skills, we give some news about the role of git in the FreeBSD project, describe the status of some tools to hunt bugs or enhance security and announce a clone of sysctl. Finally, those who are more experienced with programming will probably be interested in the great work that has been done with drivers: in particular, an aknowledgement is due to Alan Somers for having started to bring up to date our FUSE implementation, which was about 11 years behind. Other important improvements include a more user-friendly experience with trackpoints and touchpads enabled by default, much low level work on graphics, many new bhyve features, updates to the linux compatibility layer, various kernel improvements. Have a nice read! -- Lorenzo Salvadore __________________________________________________________________ FreeBSD Team Reports * Continuous Integration * FreeBSD Core Team * FreeBSD Graphics Team status report * IRC Admin * Ports Collection * Release Engineering Team Projects * bhyve - Live Migration * bhyve - Save/Restore * BIO_DELETE support for the swap pager * ENA FreeBSD Driver Update * FreeBSD SDIO and Broadcom FullMAC WiFi Support * FUSE * Fuzzing FreeBSD with syzkaller * Kernel ZLIB Update * Linux compatibility layer update * Lock-less delayed invalidation for amd64 pmap * Locking changes for vnodes during execve(2) * Mellanox Drivers Update * NFSv4.2 client/server implementation for FreeBSD * NUMA awareness in the FreeBSD kernel Architectures * Broadcom ARM64 SoC support * NXP ARM64 SoC support Third-Party Projects * Aberdeen Hackathon * Bring more Security Intelligence to FreeBSD * libvdsk - QCOW2 implementation * nsysctl 1.0 __________________________________________________________________ FreeBSD Team Reports Entries from the various official and semi-official teams, as found in the Administration Page. Continuous Integration Links FreeBSD Jenkins Instance URL: https://ci.FreeBSD.org FreeBSD CI artifact archive URL: https://artifact.ci.FreeBSD.org/ FreeBSD Jenkins wiki URL: https://wiki.freebsd.org/Jenkins freebsd-testing Mailing List URL: https://lists.FreeBSD.org/mailman/listinfo/freebsd-testing freebsd-ci Repository URL: https://github.com/freebsd/freebsd-ci Tickets related to freebsd-testing@ URL: https://preview.tinyurl.com/y9maauwg Hosted CI wiki URL: https://wiki.freebsd.org/HostedCI FreeBSD CI weekly report URL: https://hackfoldr.org/freebsd-ci-report/ Contact: Jenkins Admin Contact: Li-Wen Hsu The FreeBSD CI team maintains continuous integration system and related tasks for the FreeBSD project. The CI system regularly checks the committed changes can be successfully built, then performs various tests and analysis of the results. The results from build jobs are archived in an artifact server, for the further testing and debugging needs. The CI team members examine the failing builds and unstable tests, and work with the experts in that area to fix the code or adjust test infrastructure. The details are of these efforts are available in the weekly CI reports. The FCP for CI policy is in "feedback" state, please provide any comments to freebsd-testing@ or other suitable lists. We had a testing working group in 201905 DevSummit Please see freebsd-testing@ related tickets for more information. Work in progress: * Fixing the failing test cases and builds * Adding drm ports building test against -CURRENT * Adding powerpc64 tests job: https://github.com/freebsd/freebsd-ci/pull/33 * Implementing automatic tests on bare metal hardware * Extending and publishing the embedded testbed * Planning for running ztest and network stack tests * Help more 3rd software get CI on FreeBSD through a hosted CI solution __________________________________________________________________ FreeBSD Core Team Contact: FreeBSD Core Team The FreeBSD Core Team is the governing body of FreeBSD. * Core approved source commit bits for Doug Moore (dougm), Chuck Silvers (chs), Brandon Bergren (bdragon), and a vendor commit bit for Scott Phillips (scottph). * The annual developer survey closed on 2019-04-02. Of the 397 developers, 243 took the survey with an average completion time of 12 minutes. The public survey closed on 2019-05-13. It was taken by 3637 users and had a 79% completion rate. A presentation of the survey results took place at BSDCan 2019. * The core team voted to appoint a working group to explore transitioning our source code 'source of truth' from Subversion to Git. Core asked Ed Maste to chair the group as Ed has been researching this topic for some time. For example, Ed gave a MeetBSD 2018 talk on the topic. There is a variety of viewpoints within core regarding where and how to host a Git repository, however core feels that Git is the prudent path forward. * The project received many Season of Docs submissions and picked a top candidate. Google will announce the accepted technical writer projects on 2019-08-06. We are hoping for lots of new and refreshed man pages. __________________________________________________________________ FreeBSD Graphics Team status report Links Project GitHub page URL: https://github.com/FreeBSDDesktop Contact: FreeBSD Graphics Team Contact: Niclas Zeising The FreeBSD X11/Graphics team maintains the lower levels of the FreeBSD graphics stack. This includes graphics drivers, graphics libraries such as the MESA OpenGL implementation, the X.org xserver with related libraries and applications, and Wayland with related libraries and applications. In the last report, half a year ago, several updates and changes had been made to the FreeBSD graphics stack. To further improve the user experience, and to improve input device handling, evdev was enabled in the default configuration in late 2018. Building on that, we have enabled IBM/Lenovo trackpoints and elantech and synaptics touchpads by default as well. The input device library libinput has been updated as the last in a series of updates bringing the userland input stack up to date. This is work that was started in 2018. We have made several improvements to the drm kernel drivers. A long-standing memory leak in the Intel (i915) driver has been fixed, and several other updates and improvements have been made to the various drm kernel driver components. A port of the drm kernel drivers using the 5.0 Linux kernel sources has been created and committed to FreeBSD ports as graphics/drm-devel-kmod. This driver requires a recent Linux KPI and is only available on recent versions of FreeBSD CURRENT. This version of the driver contains several development improvements. The generic drm (drm.ko) driver as well as the i915 (i915kms.ko) driver can now be unloaded and reloaded to ease in development and testing. This causes issues with the virtual consoles, however, so an SSH connection is recommended. To aid debugging i915kms.ko use of debugfs has been improved, but there are still limitations preventing it from being fully functional. Since debugfs is based on pseudofs it is possible that this will prevent a fully functional debugfs in its current state, so we might have to look into adding the required functionality to pseudofs or use another framework. The new in-kernel drm driver for VirtualBox, vboxvideo.ko has been ported from Linux. Support is currently an experimental work in progress. For example the virtual console won't update after loading the driver, but X- and Wayland-based compositors are working. Mesa has been updated to 18.3.2 and switched from using devel/llvm60 to use the Ports default version of llvm, currently devel/llvm80. Several userland Xorg drivers, applications, and libraries have been updated, and other improvements to the various userland components that make up the Graphics Stack have been made. We have also continued our regularly scheduled bi-weekly meetings, although work remains in sending out timely meeting minutes afterwards. People who are interested in helping out can find us on the x11@FreeBSD.org mailing list, or on our gitter chat: https://gitter.im/FreeBSDDesktop/Lobby. We are also available in #freebsd-xorg on EFNet. We also have a team area on GitHub where our work repositories can be found: https://github.com/FreeBSDDesktop __________________________________________________________________ IRC Admin Contact: IRC Admin The FreeBSD IRC Admin team manages the FreeBSD Project's presence and activity on the freenode IRC network, looking after: * Registration and management of channels within the official namespace (#freebsd*) * Channel moderation * Liaising with freenode staff * Allocating freebsd/* hostmask cloaks for users * General user support relating to channel management While the FreeBSD Project does not currently endorse IRC as an official support channel (see here and here), as it has not been able to guarantee a consistent or positive user experience, IRC Admin has been working toward creating a high quality experience, by standardising channel administration and moderation expectations, and ensuring the projects ability to manage all channels within its namespace. In the last quarter, IRC Admin: * Cleaned up (deregistered) registrations for channels that were defunct, stale, out of date, or had founders that were inactive (not seen for > 1 year). Channels that were found to be otherwise active have been retained. FreeBSD now has ~40 channels registered from a previous total of over 150. * Documented baseline configuration settings in the Wiki for channels, including ChanServ settings, channel modes, registration policy, etc. * Established multiple documented methods for reporting user abuse or other channel issues to IRC Admin for resolution Upcoming changes: * Work with existing #freebsd* channels to standardise channel management, settings and access. * Migrate, forward and/or consolidate existing or duplicate #freebsd* channels to channels with a standard naming convention. * Work with unofficial ##freebsd* channels to migrate them to the official #freebsd* channels if suitable * Update existing IRC-related website and documentation sources the describe the official state of project managed IRC presence on freenode. Lastly, and to repeat a previous call, while the vast majority of the broader user community interacts on the freenode IRC network, the FreeBSD developer presence still needs to be significantly improved on freenode. There are many opportunities to be had by increasing the amount and quality of interaction between FreeBSD users and developers, both in terms of developers keeping their finger on the pulse of the community and in encouraging and cultivating greater contributions to the Project over the long term. It is critical to have a strong developer presence amongst users, and IRC Admin would like again to call on all developers to join the FreeBSD freenode channels to increase that presence. Users are invited to /join #freebsd-irc on the freenode IRC network if they have questions, ideas, constructive criticism, and feedback on how the FreeBSD Project can improve the service and experience it provides to the community on IRC. __________________________________________________________________ Ports Collection Links About FreeBSD Ports URL: https://www.FreeBSD.org/ports/ Contributing to Ports URL: https://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing/= ports-contributing.html FreeBSD Ports Monitoring URL: http://portsmon.freebsd.org/index.html Ports Management Team: URL: https://www.freebsd.org/portmgr/index.html Contact: Ren=E9 Ladan Contact: FreeBSD Ports Management Team The following was done during the last quarter by portmgr to keep things in the Ports Tree going: During the last quarter the number of ports rose to just under 37,000. At the end of the quarter, there were 2146 open PRs and 7837 commits (excluding 499 on the quarterly branch) from 172 committers. This shows a slight decrease in activity compared to previous quarter. People come and go, last quarter we welcomed Pedro Giffuni (pfg@), Piotr Kubaj (pkubaj@) and Hans Petter Selasky (hselasky@). Pedro and Hans Petter were already active as src committers. We said goodbye to gordon@, kan@, tobez@, and wosch@. On the infrastructure side, a new USES=3Dcabal was introduced and various default versions were updated: MySQL to 5.7, Python to 3.6, Ruby to 2.5, Samba to 4.8 and Julia gained a default version of 1.0. The web browsers were also updated: Firefox to 68.0 and Chromium to 75.0.3770.100 During the last quarter, antoine@ ran a total of 41 exp-runs to test various package updates, bump the stack protector level to "strong", switch the default Python version to 3.6 as opposed to 2.7, remove sys/dir.h from base which has been deprecated for over 20 years, and convert all Go ports to USES=3Dgo. __________________________________________________________________ Release Engineering Team Links FreeBSD 11.3-RELEASE schedule URL: https://www.freebsd.org/releases/11.3R/schedule.html FreeBSD 11.3-RELEASE announcement URL: https://www.freebsd.org/releases/11.3R/announce.html FreeBSD 12.1-RELEASE schedule URL: https://www.freebsd.org/releases/12.1R/schedule.html FreeBSD development snapshots URL: https://download.freebsd.org/ftp/snapshots/ISO-IMAGES/ Contact: FreeBSD Release Engineering Team The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes and maintaining the respective branches, among other things. During the second quarter of 2019, the FreeBSD Release Engineering team started the 11.3-RELEASE cycle, with the code slush starting May 3rd. Throughout the cycle, there were three BETA builds and three RC builds, all of which in line with the originally-published schedule. The final RC build started June 28th, with the final release build targeted for July 5th. FreeBSD 11.3-RELEASE will be the fourth release from the stable/11 branch, building on the stability and reliability of 11.2-RELEASE. The FreeBSD Release Engineering Team also published the schedule for the 12.1-RELEASE, targeted to start September 6th. One important thing to note regarding the published schedule is it excludes a hard freeze on the stable/12 branch, as a test run for eliminating code freezes entirely during a release cycle. Commits to what will be the releng/12.1 branch will still require explicit approval from the Release Engineering Team, however. Additionally throughout the quarter, several development snapshots builds were released for the head, stable/12, and stable/11 branches. Much of this work was sponsored by the FreeBSD Foundation and Rubicon Communications, LLC (Netgate). __________________________________________________________________ Projects Projects that span multiple categories, from the kernel and userspace to the Ports Collection or external projects. bhyve - Live Migration Links Github wiki - How to Live and Warm Migrate a bhyve guest URL: https://github.com/FreeBSD-UPB/freebsd/wiki/Virtual-Machine-Migrat= ion-using-bhyve Github - Warm Migration branch URL: https://github.com/FreeBSD-UPB/freebsd/tree/projects/bhyve_migrati= on Github - Live Migration branch URL: https://github.com/FreeBSD-UPB/freebsd/tree/projects/bhyve_migrati= on_dev Contact: Elena Mihailescu Contact: Darius Mihai Contact: Mihai Carabas The Migration feature uses the Save/Restore feature to migrate a bhyve guest from a FreeBSD host to another FreeBSD host. To migrate a bhyve guest, one needs to start an empty guest on the destination host from a shared guest image using the bhyve tool with the -R option followed by the source host IP and the port to listen to migration request. On the source host, the migration is started by executing the bhyvectl command with the --migrate or --migrate-live option, followed by the destination host IP and the port to send to the messages. New features added: * Clear the dirty bit after each migration round * Extend live migration to highmem segment Future tasks: * Refactor live migration branch * Rebase live migration * Extend live migration to unwired memory This project was sponsored by Matthew Grooms. __________________________________________________________________ bhyve - Save/Restore Links Github repository for the snapshot feature for bhyve URL: https://github.com/FreeBSD-UPB/freebsd/tree/projects/bhyve_snapshot Github wiki - How to Save and Restore a bhyve guest URL: https://github.com/FreeBSD-UPB/freebsd/wiki/Save-and-Restore-a-virtual-m= achine-using-bhyve Github wiki - Suspend/resume test matrix URL: https://github.com/FreeBSD-UPB/freebsd/wiki/Suspend-Resume-test-ma= trix Phabricator review - bhyve Snapshot Save and Restore URL: https://reviews.freebsd.org/D19495 Contact: Elena Mihailescu Contact: Darius Mihai Contact: Mihai Carabas The Save/Restore for bhyve feature is a suspend and resume facility added to the FreeBSD/amd64's hypervisor, bhyve. The bhyvectl tool is used to save the guest state in three files (a file for the guest memory, a file for the states of various devices and the state of the CPU, and another one for some metadata that is used in the restore process). To suspend a bhyve guest, the bhyvectl tool must be run with the --suspend option followed by the guest name. To restore a bhyve guest from a checkpoint, one simply has to add the -r option followed by the main state file (the same file that was given to the --suspend option for bhyvectl) when starting the VM. New features added: * Open ticket on Phabricator * Apply feedback received from community Future tasks: * Add suspend/resume support for nvme * Add suspend/resume support for virtio-console * Add suspend/resume support for virtio-scsi * Add TSC offsetting for restore for AMD CPUs This project was sponsored by Matthew Grooms. __________________________________________________________________ BIO_DELETE support for the swap pager Contact: Doug Moore Contact: Alan Cox Contact: Mark Johnston An ongoing project aims to teach the swap pager to send SCSI UNMAP or ATA TRIM commands to the swap device when a block of swap space has been freed, for example when the application owning that block is exiting. SSDs have become commonplace and feature low latency for random I/O requests. This makes them appealing for use as swap devices, since lower latencies mean that applications spend less time blocked while waiting for a page-in from the swap device. To maximize write performance, some SSDs require the operating system to send a notification to the disk when a sector is no longer in use; this helps the disk optimize their usage of NAND flash cells. In FreeBSD such a notification is called a BIO_DELETE. FreeBSD's UFS and ZFS filesystems have for a long time been able to transmit BIO_DELETE requests to the devices backing the filesystem. For example, for UFS this support is enabled by specifying -t in newfs(8) or tunefs(8)'s parameters. However, FreeBSD has historically not had a corresponding implementation for swap devices. Thanks to Doug Moore, as of r349286 in -CURRENT and r349930 in stable/12 swapon(8) can send BIO_DELETE to all blocks on the specified device immediately prior to configuring it as a swap device. This is enabled by specifying -E in the swapon(8) parameters, or by adding the "trimonce" option to the swap device's /etc/fstab entry. Some in-progress work on the swap pager implements online block deletion, in which BIO_DELETE is transmitted for blocks as they are freed by applications; this will hopefully be implemented in FreeBSD 13.0. __________________________________________________________________ ENA FreeBSD Driver Update Links ENA README URL: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/ena/R= EADME Contact: Michal Krawczyk Contact: Maciej Bielski Contact: Marcin Wojtas ENA (Elastic Network Adapter) is the smart NIC available in the virtualized environment of Amazon Web Services (AWS). The ENA driver supports multiple transmit and receive queues and can handle up to 100 Gb/s of network traffic, depending on the instance type on which it is used. ENAv2 has been under development for FreeBSD, similar to Linux and DPDK. Since the last update internal review and improvements of the patches were done, followed by validation on various AWS instances. Completed since the last update: * Upstream of the ENAv2 patches - revisions r348383 - r348416 introduce a major driver upgrade to version v2.0.0. Along with various fixes and improvements, the most significant features are LLQ (Low Latency Queues) and independent queues reconfiguration using sysctl commands. * Implement NETMAP support for ENA Todo: * Internal review and upstream of NETMAP support This project was sponsored by Amazon.com Inc. __________________________________________________________________ FreeBSD SDIO and Broadcom FullMAC WiFi Support Links FreeBSD Wiki SDIO page URL: https://wiki.freebsd.org/SDIO Contact: Bjoern Zeeb SDIO is an interface designed as an extension to SD Cards to allow attachments of various other peripherals, e.g., WiFi or Bluetooth. Work has been ongoing by Ilya Bakulin on the MMCCAM stack to provide the infrastructure to be able to have SD cards and SDIO devices attached side-by-side facilitating FreeBSD's CAM framework. Based on this excellent work over the last years, SDIO support was finished earlier this year and committed to FreeBSD HEAD with the intention to merge to 12 at a later time. Facilitating the newly available SDIO bus, work started to port Broadcom's FullMAC WiFi driver. This work is still in progress and expected to complete later this year. With this WiFi support for the Raspberry Pi and other embedded boards will become available. Likewise drivers for other SDIO devices can be developed now. This project was sponsored by The FreeBSD Foundation. __________________________________________________________________ FUSE Contact: Alan Somers FUSE (File system in USErspace) allows a userspace program to implement a file system. It is widely used to support out-of-tree file systems like NTFS, as well as for exotic pseudo file systems like sshfs. FreeBSD's fuse driver was added as a GSoC project in 2012. Since that time, it has been largely neglected. The FUSE software is buggy and out-of-date. Our implementation is about 11 years behind. During Q2 I nearly finished the FUSE overhaul that I begain in Q1. I raised the protocol level from 7.8 to 7.23, fixed many bugs (see 199934, 216391, 233783, 234581, 235773, 235774, 235775, 236226, 236231, 236236, 239291, 236329, 236379, 236381, 236405, 236327, 236466, 236472, 236473, 236474, 236530, 236557, 236560, 236647, 236844, 237052, 237181, 237588, and 238565), and added the following features: * Optional kernel-side permissions checks (`-o default_permissions`) * Implement VOP_MKNOD, VOP_BMAP, and VOP_ADVLOCK * Allow interrupting FUSE operations * Support named pipes and unix-domain sockets in fusefs file systems * Forward UTIME_NOW during utimensat(2) to the daemon * kqueue support for /dev/fuse * Allow updating mounts with mount -u * Allow exporting fusefs file systems over NFS * Server-initiated invalidation of the name cache or data cache * Respect RLIMIT_FSIZE * Try to support servers as old as protocol 7.4 I also added the following performance enhancements: * Implement FUSE's FOPEN_KEEP_CACHE and FUSE_ASYNC_READ flags * Cache file attributes * Cache lookup entries, both positive and negative * Server-selectable cache modes: writethrough, writeback, or uncached * Write clustering * Readahead * Use counter(9) for statistical reporting All that remains is to finish merging the branch, and deal with any newly introduced bugs. This project was sponsored by The FreeBSD Foundation. __________________________________________________________________ Fuzzing FreeBSD with syzkaller Links syzkaller URL: https://github.com/google/syzkaller Contact: Mark Johnston Contact: Andrew Turner Contact: Michael Tuexen Contact: Ed Maste See the syzkaller entry in the 2019q1 quarterly report for an introduction to syzkaller. syzkaller continues to find FreeBSD kernel bugs. A number of such bugs have been fixed in the past quarter, and we continue to investigate and fix bug reports from syzkaller. Work to extend syzkaller's capabilites has progressed: Andrew Turner has implemented support for fuzzing the 32-bit compatibility layer in amd64 kernels, helping illuminate some of the darker corners of the kernel, and it is now possible to use bhyve as a VM backend to syzkaller, so it is now efficient and convenient to fuzz FreeBSD on FreeBSD. Some planned work includes: enabling the use of ZFS as the base filesystem for fuzzer VMs; extending the range of system calls and ioctls covered by syzkaller; enabling LLVM sanitizers in the kernel so as to catch more issues; and making use of netdump(4) to capture kernel dumps for panics found by syzkaller, making it much easier to diagnose bugs for which syzkaller was unable to find a reproducible test case. This project was sponsored by The FreeBSD Foundation. __________________________________________________________________ Kernel ZLIB Update Links Review D19706 URL: https://reviews.freebsd.org/D19706 Contact: Yoshihiro Ota Kernel zlib upgrade is in progress. Xin (delphij@) and I have been working closely for zlib upgrade. We relocated contrib/zlib to sys/contrib/zlib in order for kernel code to access zlib in the tree. We also deleted dead code that depended on zlib and inflate - inflate is a fork of unzip to uncompress gzip files. We also renamed crc.h to avoid conflicts with zlib/crc.h. Next goal is to compile both old zlib and new zlib into the kernel allowing to switch each zlib user independently. __________________________________________________________________ Linux compatibility layer update Contact: Edward Tomasz Napierala The project aims to improve the Linux compatibility layer, to make it more compatible with recent Linux releases, and also to lower the bar for potential developers who want to start contributing to it. The initial effort focused on tooling, to make it easier to debug problems and to prevent future regressions. The first part involved making it possible to use Linux strace(1) utility and providing it as linux-c7-strace package. The reason is that while FreeBSD truss(1) and ktrace(1) can trace Linux binaries, they cannot decode Linux-specific flags and structures. The second part involved providing Linux Test Project binaries as linux-ltp package. There is ongoing work to hook it up to the FreeBSD CI infrastructure http://ci.FreeBSD.org. There was also a number of improvements and fixes to bugs discovered in the process. One of them (not yet committed) fixes binaries linked against newer version of libc, effectively unbreaking binaries from recent Ubuntu releases. This project was sponsored by FreeBSD Foundation. __________________________________________________________________ Lock-less delayed invalidation for amd64 pmap Contact: Konstantin Belousov The Virtual Memory machine-dependent layer (pmap) on amd64 needs to track all mappings for the managed physical memory pages, to be able to either destroy all of them (for page-out), or change them from writeable to read-only (e.g. to sync the page content to file, without racing with modifications through user writes). The mappings are accounted by creating pv_entry which records the address space (implicitly, by linking the pv entry to pmap) and the virtual address of the mapping. Previous work split the lock protecting the pv entries lists from other VM locks into the pvh_global_lock lock, which was global for all address spaces. You can see it in i386 pmap.c still. Later, hashed per-page pv lists locks were introduced, which would reduce contention on pv lists maninulations for different pages, but unfortunately the pvh_global_lock was still needed to guarantee the safety of some operations. Problem arises because amd64 pmap uses pmap lock to protect page tables and TLB consistency, which is per-pmap locks different from pv lists locks. When updating page table entry, we never drop pmap lock until the necessary TLB invalidation is done globally, including signalling other CPUs with IPI. But pv list locks can be unlocked before the necessary invalidation is done. So for instance when pmap is asked to remove all mappings of the specific page (pmap_remove_all(9)), it checks pv list of the page to find the mappings. The list might appear empty despite other CPUs TLB were not yet invalidated. If such page is reused, other CPUs might change its content using cached TLB entries. Allowing that means allowing both silent data corruption and opening security hole. So the global pvh lock was held until all pmaps invalidated their TLBs. This mechanism has obvious scalability issues, and instead a generation-count based scheme for handling delayed invalidation (DI) was developed, where each thread that might remove entry from pv list acquired a generation number and marked the page with it, see pmap_delayed_invl_page(9). Then, on e.g. pmap_remove_all(9) or pmap_remove_write(9), pmap code waits for the maximum current thread's invalidation generation number to pass the page's generation, which guarantees that all required TLB invalidations are done. Original implementation of DI allowed to get rid of pvh_global_lock, and only used a private mutex to handle sequential queueing of the coming and leaving threads, protecting a bounded region. A problem with that appeared e.g. in scalability benchmarks which did massive parallel unmaps, causing most of the threads to contend on DI queueing. Current implementation of DI switched to lock-less queue algorithm using the approach proposed by T.L. Harris and relying on double-CAS to coalesce generation count and queueing. It uses ifuncs to select either previous locked DI or current lock-less implementation, only old AMD Athlons which did not implemented the CMPXCHG16B instruction falls to the locked implementation by default. Lock-less implementation still blocks the waiting thread on turnstile to avoid priority-inversion issues, but practically the wait occur very rare, typical parallel buildworld generates single-digit number of the events. The patch got a lot of testing from Peter Holm, continuous reviews by Mark Johnston while I worked out bugs and live-lock problems in the implementation, and additional testing by Mateusz Guzik who helped to identify a priority inversion bug with the wait. This project was sponsored by The FreeBSD Foundation. __________________________________________________________________ Locking changes for vnodes during execve(2) Contact: Konstantin Belousov The execve(2) family of syscalls replaces the executing image in the current process. The file containing the program text, data, and arbitrary other pre-initialized segments for the newly activated image is usually called the text file. FreeBSD marks the text file as such, the mark is mutually exclusive with any opening of the file for write. In other words, file opened for write cannot be executed, and text file cannot be opened for write. During the execve(2) syscall processing, kernel needs to lock the text file' vnode. This is done both to satisfy the VFS calls protocol, and to ensure that there is no incompatible parallel changes occuring to the text vnode. A vnode can be locked either in exclusive mode, which is mutually incompatible with any other lock acquisition, or in shared mode, which is only incompatible with exclusive requests, but allows other shared owners. In principle, there is no reason why would execve(2) need an exclusive vnode lock, since it does not modify neither content nor metadata for the text vnode. The only exception is the marking of the vnode as text, which was done using VV_TEXT flag in v_vflag and protected by the vnode lock. Since we modify v_vflag, the vnode lock protecting the modification should be taken exclusive. The end result is that execve(2)'s of the same file are serialized. For instance, if user runs parallel build, which executes more than one job for compiling, all invocation of the compiler are serialized during execve(2). The count of opens for write is contained in other struct vnode member named v_writecount, which was protected by the vnode lock as well. Since text is mutually exclusive with an open for write, I reused v_writecount to indicate text references. Now, negative v_writecount counts the number of text references. The v_writecount content is literally protected by the vnode interlock, but normally all mutators also own vnode lock at least in the shared mode. This way, we no longer need to acquire exclusive text vnode lock during execve(2), removing the serializing point. Additional positive effect is that we started to account the precise number of text references on the vnode. Before, we cleared VV_TEXT on the last unmap of the text vnode, potentially allowing obscure DoS where mapping the text file while it is executed prevented writes until the mapping is destroyed. Now we mark the mappings for text explicitly in the vm_map_entry and dereference v_writecount by +1 when such entry is unmapped. This project was sponsored by The FreeBSD Foundation. __________________________________________________________________ Mellanox Drivers Update Links Mellanox OFED for FreeBSD Documentation URL: http://www.mellanox.com/page/products_dyn?product_family=3D193&mta= g=3Dfreebsd_driver Contact: Slava Shwartsman Hans Petter Selasky Konstantin Belousov The mlx5 driver provides support for ConnectX-4 [Lx], ConnectX-5 [Ex] and ConnectX-6 [Dx] adapter cards. The mlx5en driver provides support for Ethernet adapter cards, whereas mlx5ib driver provides support for InfiniBand adapters and RDMA over Converged Ethernet (RoCE). Following updates done in mlx5 drivers: * 200Gb/s ConnectX-6 Ethernet: Added support for Mellanox Socket Direct Adapters which allows, among the rest of the capabilities, to run up to 200Gb/s on a PCIe Gen 3.0 on a LAG interface. * Support for "BlueField" - Multicore System On A Chip: Added support for RShim driver for BlueField Multicore System On A Chip(SOC). The RShim driver provides access to the RShim resources on the BlueField target accessible from an external host machine. The current RShim version provides device files for boot image push and virtual console access. It also creates virtual network interface to connect to the BlueField target and provides access to internal RShim registers. * Firmware Burning and Diagnostics Tools: Added MSTFLINT to ports, this package contains a burning and diagnostic tools for Mellanox NICs. This package contains following tools: mstflint - Tools which allows to query and burn firmware. mstconfig - This tool queries and sets non-volatile configurable options for Mellanox HCAs. mstregdump - This utility dumps hardware registers from Mellanox hardware. mstmcra - This debug utility reads/writes a to/from the device configuration register space. mstvpd - This utility dumps the on-card VPD. and more. * OFED-FreeBSD-v3.5.1 Upstream: Pushed upstream and MFCed OFED-FreeBSD-v3.5.1 driver - more details on the content of this update can be found in Mellanox OFED for FreeBSD documentation page. General updates: * Submitted papers for EuroBSDcon for a joint talk with Netflix titled "Kernel TLS and TLS Hardware Offload". The papers were accepted. * Mellanox is intensively working to improve its cooperation with the FreeBSD community. As part of this effort, FreeBSD users are invited to propose features and enhancements to further develop and enrich the end-user experience. In addition, Mellanox continues to identify and present the right solutions to meet customers' needs. This project was sponsored by Mellanox Technologies. __________________________________________________________________ NFSv4.2 client/server implementation for FreeBSD Links current sources URL: https://svnweb.freebsd.org/base/projects/nfsv42 Contact: Rick Macklem NFSv4.2 is a newer minor version of NFSv4, made up of a set of optional operations/features. A majority of these operations are related to the POSIX operations posix_fadvise(2), posix_fallocate(2) and lseek(2)'s support for SEEKHOLE/SEEKDATA. There is also a Copy operation that allows a byte range of a file to be copied to another file locally on the NFS server, avoiding data transfer over the wire in both directions. FreeBSD-current now has a Linux compatible copy_file_range(2) syscall that will invoke this Copy operation on NFSv4.2 mounts. There is also support for MAC labelling, but it requires changes to the RPCSEC_GSS implementation to add V3 support and, as such, may not happen soon. The implementation of NFSv4.2 (RFC-7862) is progressing nicely. At this time, the LayoutError, IOAdvise, Allocate and Copy operations have been implemented. There is still work to be done on Copy, to add asynchronous support, so that large copies do not result in a long delay for the RPC's reply. The major operation that will be implemented next is Seek, so that lseek(SEEKHOLE/SEEKDATA) will work for the NFSv4.2 mounts. It is hoped that this implementation will be ready for FreeBSD-current/head in time for the FreeBSD-13 release. Testing is always appreciated and can be done by downloading the modified kernel from the svn repository in base/rojects/nfsv42 and then building and testing it on a couple of recent FreeBSD-current systems. If anyone is conversant with Kerberos and wants to take on the challenge of adding RPCSEC_GSS_V3 support to the kernel RPC, a patch that does that would also be greatly appreciated. __________________________________________________________________ NUMA awareness in the FreeBSD kernel Contact: Jeff Roberson Contact: Andrew Gallatin Contact: Mark Johnston A set of patches to improve the state of NUMA awareness in the FreeBSD kernel are being developed and refined. This work also aims to generally improve the performance of FreeBSD's memory management subsystem on systems with many CPUs. FreeBSD 12.0 featured a number of large changes which improve its performance on systems with a non-uniform memory architecture. That is, systems in which memory access latency for a given address varies depending on the CPU. Another round of improvements is being developed and will soon be available in FreeBSD-CURRENT. Short descriptions of some of these patches follow; a few have already been committed to FreeBSD-CURRENT. In FreeBSD terminology, a memory page whose contents may not be evicted is referred to as "wired." Pages may be wired under different circumstances: for instance, all kernel memory is wired, and userland applications may request that ranges of memory be wired using the mlock(2) and mlockall(2) system calls. FreeBSD has historically defined a system-wide limit on the number of wired pages so as to avoid deadlocks that may arise when too much of a system's memory cannot be reclaimed to satisfy new memory allocations. This limit was applied only to userland wiring requests, but kernel wirings were counted against the limit, so a large source of kernel wirings could cause mlock(2) failures. This occurs frequently with a large ZFS ARC, for example. In FreeBSD-CURRENT this limit has been changed such that only userland wirings are counted against the limit; the kernel contains a number of mechanisms to apply back-pressure to kernel memory usage, so the use of a global limit on all wirings did not provide much benefit. This fixes a common problem on large ZFS systems, and helps enable some other architectural improvements to the code which manages page wirings. FreeBSD has historically maintained two separate reference counters in the structure which describes a single physical page of memory. These counters initially had quite different properties, but have over time become more and more similar. Some work to merge the two counters has landed in FreeBSD-CURRENT. This does not have any user-visible effects, but it simplifies the page management code and removes a large amount of code which existed solely to transform references of one type to the other. Such code also made use of heavily contended locks, so the simplification improved kernel scalability for some workloads and has enabled further scalability improvements. UMA is the slab allocator used in FreeBSD's kernel. It is the backend which services virtually all dynamic memory allocations performed in the kernel. The first round of NUMA improvements added NUMA awareness to the "keg" layer of UMA, which allocates and manages slabs. However, the frontend of UMA, which provides several layers of caching for objects, did not provide domain-aware caching, so over time the caches would become "polluted" with objects from different memory domains. However, this caching layer is being modified to ensure that objects from different memory domains are partitioned, helping ensure that consumers can perform domain-local allocations and frees efficiently. This will enable a global "first-touch" allocation policy for UMA-managed objects. During boot, the FreeBSD kernel allocates a number of static data structures to track physical memory. These structures have historically lived in the lowest available range of physical memory, so they many not inhabit the same NUMA domain as the memory that they track. This is suboptimal when one tries to affinitize a workload to a particular NUMA domain: if while executing the workload the kernel frequently accesses page structures for local memory, and the page structures themselves are not placed in local memory, the kernel will perform many remote memory accesses. Some in-progress work for the amd64 platform creates multiple arrays of page tracking structures, one per NUMA domain, and ensures that each array is local to its domain. This complicates the task of initializing kernel data structures during boot, but can substantially reduce the amount of cross-domain communication that occurs while the kernel is performing useful work. Similarly, some patches to affinitize per-CPU structures are being developed; while most per-CPU memory allocations already return CPU-local memory, some structures allocated during boot are not yet properly placed with respect to the accessing CPU's memory domain. This project was sponsored by Netflix. __________________________________________________________________ Architectures Updating platform-specific features and bringing in support for new hardware platforms. Broadcom ARM64 SoC support Contact: Michal Stanek Contact: Kornel Duleba Contact: Marcin Wojtas The Semihalf team continued working on FreeBSD support for the Broadcom BCM5871X SoC series BCM5871X are quad-core 64-bit ARMv8 Cortex-A57 communication processors targeted for networking applications such as 10G routers, gateways, control plane processing and NAS. Completed since the last update: * iProc PCIe root complex (internal and external buses): fixes and improvements, including adding a BCM58712 quirk to GICv2m driver * BNXT Ethernet support: sys/dev/bnxt.c driver has been extended to support the BCM58700 variant, and the iflib was made to work without IO cache coherency In progress: * Crypto engine acceleration for IPsec offloading. Todo: * Upstreaming of work. This work is expected to be submitted/merged to HEAD in the second half of 2019. This project was sponsored by Juniper Networks, Inc. __________________________________________________________________ NXP ARM64 SoC support Contact: Marcin Wojtas Contact: Artur Rojek The Semihalf team initiated working on FreeBSD support for the NXP LS1046A SoC LS1046A are quad-core 64-bit ARMv8 Cortex-A72 processors with integrated packet processing acceleration and high speed peripherals including 10 Gb Ethernet, PCIe 3.0, SATA 3.0 and USB 3.0 for a wide range of networking, storage, security and industrial applications. Already completed: * Platform base support (ramp-up multi-user SMP operation with UART) * SATA 3.0 In progress: * USB3.0 * SD/MMC * I2C Todo: * Ethernet support * GPIO * QSPI * Upstreaming of developed features. This work is expected to be submitted/merged to HEAD in the Q4 of 2019. This project was sponsored by Alstom Group. __________________________________________________________________ Third-Party Projects Many projects build upon FreeBSD or incorporate components of FreeBSD into their project. As these projects may be of interest to the broader FreeBSD community, we sometimes include brief updates submitted by these projects in our quarterly report. The FreeBSD project makes no representation as to the accuracy or veracity of any claims in these submissions. Aberdeen Hackathon At BSDCam in Cambridge last year we had a discussion to create a template Hackathon in the same way we have a template for Devsummits. To test out the idea I was convinced (I swear tricked is the correct word) to host a Hackathon in Aberdeen. As a project I think we benefit a lot from hackathons, but they do take a little organisation. The worst part of this is dealing with getting money from attendees so you can pay for events. I spoke with Deb Goodkin from the foundation at BSDCam and we arranged to use their new EventBrite based system to handle ticketing. Overall this system made it straight forward for attendees to register and get me their details and requirements. After the event the expenses were then recouped from the foundation. This was much easier than me putting together a custom system or even setting up and using EventBrite myself. The hackathon went well, you can read in Benedict and Kristof's reports that follow, but it was less well attended than I originally expected. For hackers planning future hackathons remember to take heed of common national holidays (we could have planned the event to not land at Easter) and expect major geopolitical events to make things unpredictable (we knew Brexit would do something, but not when). I need to thank the University of Aberdeen for providing the location for the Hackathon and to encourage you to run a hackathon where you are. The next one should be in your home town. Benedict Reuschling The hackathon in Aberdeen was happening in the week of Easter at the University of Aberdeen. Although only Kristof Provost (kp@) and myself joined our host Tom Jones, I still consider it a productive week for us. The overall theme of the hackathon was networking and each of us provided something towards that goal (be it PRs, submitting unfinished work, or other bits and pieces). We got together the night of Tuesday, April 16 over dinner and talked about what our plans were for the week. Kristof and I had talked at AsiaBSDcon when I took his tutorial about Testing in FreeBSD that we should add a chapter about it in the developers handbook. We also used our first meeting to synchronize each other about the latest news in FreeBSD from our developers viewpoint. The next day, we met up at the Frazer Noble building where the hackathon was taking place. It was one of the newer buildings on campus, nicely integrated into the older houses of the city. Since we were only a handful, we sat in Tom's office for the hackathon, which had plenty of room. He also showed us the room where we are supposed to be having the hackathon if we were more people and Tom gave us a little tour. Working in a university myself, I'm always interested in how other education organizations are structured and the rooms and equipment they provide for learning. Overall, my impression was that there is a good amount of space and equipment available, which we could have used in the hackathon. After returning, we decided to use a special tag in the commits we would be doing to identify them as coming from this hackathon. We chose "Event:" for it as it is a general enough term to be used at other events like conferences, too. The "Sponsored by:" line we used in the past is more for companies or individuals sponsoring certain features, so I created a review to add this line to the committers guide. Kristof had a couple of changes to the pf chapter in the FreeBSD handbook for me, so I started going through those. I created a review for him and the commit was made there and then, making use of the short feedback cycle. Originally, we thought about bringing in people via hangouts, but then resolved to contact people via our usual IRC channel if we needed their input. Kristof and Tom worked on some network specific stuff, whereas I started work on creating an initial draft for the testing chapter. We would occasionally start talking about something and then return to our work in silence. If we needed to coordinate or had questions, we simply asked and could continue once we got our answer. This provided a nice atmosphere to work in. I tackled some doc PRs while Kristof found a bug in pf and fixed it. The afternoons were spent at different locations within walking distance. Tom made sure we got a good impression on how it is to be a student and that there is both taste and variety of food available. In the evenings, Tom drove us into town to have dinner at various restaurants over the week. Aberdeen has a lot to offer as a city. Starting from the second day, Kristof and I would meet up at my hotel, which was close to the Aberdeen beach and walk along it to the University. According to Tom, it is possible to see Dolphins when the weather is right and the gulf stream provides the city with enough warmth that the winters aren't as bad as you'd think this far up north. Tom also gave us a tour of the zoological department of the university, which offered a beautiful garden with various plants and trees, as well as a museum with zoological specimen. This offered a great spot for photographs and to unwind a bit from the technical discussions we've had. Tom also had t-shirts made for the event, which are already rare collectors items. I had to return on Sunday, so Tom took us on a tour of the Scottish highlands in his car the day before. We stopped at a couple of places to take pictures and Tom would explain at lot to us having lived there all his life. We came to Stonehaven and had fish and chips there from a take-out restaurant that had a lot of awards for sustainable fishing. This was certainly a highlight for the week and even then, we couldn't stop talking about FreeBSD and networking. Although more people would maybe have produced more output, the three of us were certainly productive as a small group. It also made planning and coordination easier and more flexible. Tom Jones had done a lot of preparation and was an excellent guide. I would encourage him to host another such hackathon in the future and hope that next time, more people will take a trip to Aberdeen to spend some time hacking on FreeBSD Kristof Provost While I'd been to Scotland before I'd never seen Aberdeen. It's a charming city, and I enthusiastically recommend visiting. I arrived a little while after Benedict, but made it to my hotel easily, and turned up in time to join Benedict and Tom for dinner. Despite being small (or perhaps because of it), the hackathon was remarkably productive. Benedict and I went through the pf documentation in the handbook, so that Benedict could rework and improve it. (Benedict's doing the work, but I'm going to take credit anyway.) Tom and I looked at the GSoC proposals and tried to find potential mentors for two promising proposals. Both of us are candidate mentors as well. We should know soon if our students are awarded slots. Tom also proposed a patch to eliminate RFC 2675 IPv6 Jumbograms. It has my enthusiastic support. I managed to look at a couple of open pf issues: * pfctl's interface_group() function checks if a name is an interface or an interface group. It still thought that interface names always ended with a number, but this assumption has been wrong for several years now. That's fixed in r346370. * The DIOCRSETTFLAGS ioctl() misused copyin() (It held a lock calling it), which could result in panics. * That previous issue was actually discovered by my local instance of syzcaller, which I'd set up to add pf support to it. That support has now been merged, so we may see more issues detected by syzcaller soon. * Also for the DIOCRSETTFLAGS problem I extended the pf tests to check for this issue. * The pf tests will now fail if the pft_set_rules call fails to set the rules. That didn't actually cause issues yet, but it'll make debugging tests slightly easier, and they may catch more problems now. On Saturday Tom took us out to discover some of the pretty bits of Scotland. It turns out there are a lot of them. I can't really do it justice, but Tom has a promising career at the Scottish tourism board when this computers fad blows over. On my way home I passed through Oslo, and took the opportunity to meet with (have lunch with) two of the EuroBSDCon local organisers. EuroBSDCon is filling up fast, make sure to register now to secure your place! __________________________________________________________________ Bring more Security Intelligence to FreeBSD Links Maltrail - distributed Malware detection URL: https://github.com/stamparm/maltrail Wazuh - thread detection and incident response URL: https://wazuh.com/ Contact: Michael Muenz To bring more Security Intelligence we maintain the FreeBSD port of zmaltail. This open source project based on Python can act as a sensor and/or as a central server. It listens in defined ports or protocols and compares IP addresses and domains against static and dynamic feeds, contributed by the community. As you can install this piece of software on multiple firewalls and let them send to a central server, you are able to detect attacks and compromises very fast. Within Q2 we updated the port to the latest version and are constantly in contact with the core developer (also co-author of SQLmap) to bring out new features. The second project we are currently trying to add as a port is Wazuh. Wazuh is a fork of Ossec which is already in the ports tree. Compared to Ossec, Wazuh has some intelligent addition like full ELK-Stack integration with own apps and dashboards. With Wazuh installed on your webserver, or even on your windows desktop you can monitor file integrity or log files for most kind of attacks. Active response features let you e.g. send API calls to your firewalls to dynamically block this offender. As Wazuh offers a complete ELK-Stack you can use it also as a central logging solution for better security insights into your network. This project was sponsored by m.a.x. Informationstechnologie AG. __________________________________________________________________ libvdsk - QCOW2 implementation Links Github - libvdsk repo URL: https://github.com/xcllnt/libvdsk Contact: Sergiu Weisz Contact: Marcel Molenaar Contact: Marcelo Araujo Contact: Mihai Carabas Add support for using QCOW in bhyve using the libvdsk library. Libvdsk was used to substitute the regular disk operations from bhyve with a call to libvdsk which will in turn call the disk-specific handler for the operation. To use this feature one has to install the libvdsk-enabled bhyve version along with libvdsk from the libvdsk repo linked above. New features added: * Extend libvdsk to make it easier to implement new formats * Improve read/write performance and stability * Add support for Copy-On-Write Future tasks: * Integrate libvdsk in bhyve Matthew Grooms __________________________________________________________________ nsysctl 1.0 Links gitlab.com/alfix/nsysctl URL: https://gitlab.com/alfix/nsysctl sysutils/nsysctl port URL: https://www.freshports.org/sysutils/nsysctl/ Tutorial URL: https://alfix.gitlab.io/bsd/2019/02/19/nsysctl-tutorial.html Contact: Alfonso Sabato Siciliano The nsysctl utility is a /sbin/sysctl clone, to get or set the kernel state, supporting libxo and extra options. nsysctl [--libxo=3Dopts [-r tagname]] [-DdFGgIilmNpqTt[V|v[h[b|o|x]]]Wy] [-e sep] [-B ] [-f filename] name[=3Dvalue[,value]] ... nsysctl [--libxo=3Dopts [-r tagname]] [-DdFGgIlmNpqTt[V|v[h[b|o|x]]]Wy] [-e sep] [-B ] -A|a|X You could use nsysctl to explore the sysctl MIB showing the value and the info of an object. The output is explicitly indicated by the options and is printed via libxo in human and machine readable formats, moreover some value is parsed to display it in a structured mode (e.g., vm.phys_free). The support for efi_map_header was added but it is untested, someone could help by trying it via machdep.efi_map. Please refer to the tutorial for a more thorough description. __________________________________________________________________ --WIyZ46R2i8wDzkSu Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGmBAEBCgCQFiEEbvjBe1hu6u1NeinjJCKD+Vwk/7oFAl1ifcRfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDZF RjhDMTdCNTg2RUVBRUQ0RDdBMjlFMzI0MjI4M0Y5NUMyNEZGQkESHHRyYXN6QGZy ZWVic2Qub3JnAAoJECQig/lcJP+68WQH+wQOBid2IwHXPyS+vxpl2YevaP+7yjmh ivNM3we+xwB5E8ETufE4ZkQvzDPXbQQ1mQp+SUbmAmoNT53XU35RPDTUH4Ine5+X mwu7arfgOU2vpr5ej0Q4rzzD9ZSYwvz4cqWeU3qqKDXwGh1rIU67hXdu2cq6nWd0 h8pR/k9CGsCt0HqbSSaBr1YSKxONASz0fhUyNykxoKCVcKhpN7ordY3kUawm1M9m fWBmIsnuRAAoBGk+mOb4H3qELTBLxI7sFoKFqNCzw15l9Cyw2nfDvtzeqB8/6CEO YylUTQ0qW/th24ODNNiZpU/6VvajPyC4Wl7Rs9uvRc8H3ebwUrrccQI= =bo0Y -----END PGP SIGNATURE----- --WIyZ46R2i8wDzkSu-- From owner-freebsd-hackers@freebsd.org Sun Aug 25 19:28:12 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9D884E77E8 for ; Sun, 25 Aug 2019 19:28:12 +0000 (UTC) (envelope-from vijaykumar9597@gmail.com) Received: from mail-qt1-x82f.google.com (mail-qt1-x82f.google.com [IPv6:2607:f8b0:4864:20::82f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46GlXq5nxxz4LYt for ; Sun, 25 Aug 2019 19:28:11 +0000 (UTC) (envelope-from vijaykumar9597@gmail.com) Received: by mail-qt1-x82f.google.com with SMTP id y26so16032952qto.4 for ; Sun, 25 Aug 2019 12:28:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=PRjkR7OWmP08G0+NJjEg6d0s30VdPxi9O8eAjybPIwA=; b=RopaGo7kC02tP8yjJJHaM8w45Sy0P++QAmQ608kNCzdkPZ0gkgHp5p5fGeUg/CTBXO QmywzN7RY0XIIjE5Wq+vw3WMao1rhykYJU2LGGl/S5rUoCGiHU/Oy7CcHUhJb3TnVcXm QRVaHVADVz6wQO5PvkOEMLh/pBrLxzzOnda6P2F4WTqGRoZESJ16xwqArwBqnFmbd9Gp 1e1GT1O+51Vm27iA1cU0N1F0Zu/yU/fmszjKdED/HsjqzR/KST13FK9ZnDafFmOX5N6o 116OQ+q0sGE4nqX5SieNyNwAM8fE7zVPqTYDQ/jwn+2FRRBifspsCtMGWoWOvwTgnj0q GB3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=PRjkR7OWmP08G0+NJjEg6d0s30VdPxi9O8eAjybPIwA=; b=DXK5FUBZ+Y+FND1melqHqDTsmh57xeC7bCayn6FX3lSf1wUIt1o5rAcfqLNXVRBufL ibHtJCsttWkvvXpbkp9POJ8WCq7EPFpEGSs2/Jp7zp/R+gP+YnVSOq3143kw2h8r7WTu aypWZC+p2GXdJ8cte10uPwzgajsHZtKCXCe7r6BVUY2Qf8npNE6LHlwMjQxx5iJASXtY vvGEWNEv4N/fgGyaGXHAbZqIKdVlh9y2A+dECq0L6iILjmReXVya9WoU9hoOBmotbL0h VaBWWQuxlya4uUWkr/lQlHr2AczuQmLMViuxdMO3uFAmtkZGL9mGFKawObivBceW9p/C 0Vig== X-Gm-Message-State: APjAAAWyDWP+5ITsPiiwuw4SbEDOkSFnp3uh988CpGL7qaTXQKMMRzsT fxWGo+PyBbayCqJJIJSUpf9z3Zm/wd3yZs3DnbWIEfwF X-Google-Smtp-Source: APXvYqxJ/o8kPjkQz6xn/tdRMDXgmVJDO2i4/KkQ+D2I7IGfFnssTiBGdeVOzX6aYH5jT8Atwu8U3bvF2UleFzooeN0= X-Received: by 2002:ac8:1419:: with SMTP id k25mr14368838qtj.339.1566761290867; Sun, 25 Aug 2019 12:28:10 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Vijay Kumar Banerjee Date: Mon, 26 Aug 2019 00:57:59 +0530 Message-ID: Subject: Re: Question regarding framebuffer driver. To: Aleksandr Rybalko Cc: freebsd-hackers@freebsd.org X-Rspamd-Queue-Id: 46GlXq5nxxz4LYt X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=RopaGo7k; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of vijaykumar9597@gmail.com designates 2607:f8b0:4864:20::82f as permitted sender) smtp.mailfrom=vijaykumar9597@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; URI_COUNT_ODD(1.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-0.995,0]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (-9.37), ipnet: 2607:f8b0::/32(-2.87), asn: 15169(-2.33), country: US(-0.05)]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[f.2.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Aug 2019 19:28:12 -0000 On Sat, Aug 24, 2019 at 12:35 AM Aleksandr Rybalko wrote: > Hi Vijay! > > vt_fb driver is a simple consumer, it just use passed info about location > of FrameBuffer in memory and its properties, like width/height/colors/etc= . > So if display keeps blank, you have to check correctness of passed data. > Or maybe even writing driver that prepare graphic subsystem to work with > FB. > > Thanks. > > Hi Aleksandr, Thanks for the reply, I checked the data and it was alright. The screen wasn't even turning on, I figured out that the problem was in pinmuxing. The problem is now solved when I ported the pinmux driver correctly. Thanks and regards, Vijay > =D0=B2=D1=82, 16 =D0=BB=D0=B8=D0=BF. 2019 =D0=BE 15:22 Vijay Kumar Banerj= ee > =D0=BF=D0=B8=D1=88=D0=B5: > >> Hello everyone, >> >> I'm working on porting the framebuffer driver to RTEMS with Beaglebone >> Black as the target device. I have have already ported the am335x_lcd, >> tda19988, fbd and VT drivers, but the screen doesn't seem to "power up". >> From the FreeBSD bootlog (12-RELEASE), I see that the screen is >> turning on after the VT initialization message, so I guess it's somethin= g >> that happens after the vt initialization that turns the screen on. >> >> So far I have ported the vt_fb and vt_core and it boots up well with the >> message : >> VT: initialize with new VT driver "fb". >> >> But the screen doesn't seem to turn on. Can someone please tell me >> or point me to the right place in code that is responsible for turning t= he >> screen on after VT initialization? >> >> Thank you, >> Vijay >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.or= g >> " >> > > > -- > WBW > ------- > Rybalko Aleksandr > > From owner-freebsd-hackers@freebsd.org Sun Aug 25 22:51:35 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2C082C7DEF for ; Sun, 25 Aug 2019 22:51:35 +0000 (UTC) (envelope-from farhan@farhan.codes) Received: from mail.farhan.codes (mail.farhan.codes [155.138.165.43]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46Gr3V1qSCz4Zwb; Sun, 25 Aug 2019 22:51:33 +0000 (UTC) (envelope-from farhan@farhan.codes) Received: from mail.farhan.codes (rainloop [172.16.0.4]) by mail.farhan.codes (Postfix) with ESMTPSA id 3C3F9166AD; Sun, 25 Aug 2019 18:50:55 -0400 (EDT) MIME-Version: 1.0 Date: Sun, 25 Aug 2019 22:50:55 +0000 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: RainLoop/1.12.1 From: "Farhan Khan" Message-ID: <0562c72f444c42097b5af75733686ff5@farhan.codes> Subject: Re: Trouble using and understanding funopen(3) To: cem@freebsd.org Cc: freebsd-hackers@freebsd.org In-Reply-To: References: <519c2fce85fe0db1cd189d2060f09a0f@farhan.codes> X-Rspamd-Queue-Id: 46Gr3V1qSCz4Zwb X-Spamd-Bar: ------- X-Spamd-Result: default: False [-7.05 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[farhan.codes:s=mail]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; DKIM_TRACE(0.00)[farhan.codes:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[farhan.codes,reject]; NEURAL_HAM_SHORT(-0.98)[-0.979,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-3.07)[ip: (-9.86), ipnet: 155.138.160.0/20(-4.93), asn: 20473(-0.52), country: US(-0.05)]; ASN(0.00)[asn:20473, ipnet:155.138.160.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Aug 2019 22:51:35 -0000 August 22, 2019 4:28 PM, "Conrad Meyer" wrote:=0A=0A> H= i Farhan,=0A> =0A> First, I'd suggest using the more portable fopencookie= (3) interface,=0A> which is similar to funopen(3).=0A> =0A> Second, read = functions return 0 to indicate end of file.=0A> =0A> Finally, the file co= okie routines are stateful. If you want to create=0A> a pseudo-FILE that = only has 10 bytes in it, you have to track the=0A> current file offset by= creating a cookie. Here is a minimal example=0A> of using a cookie.=0A> = =0A> struct my_file {=0A> off_t offset;=0A> };=0A> =0A> my_read(void *v, = buf, len)=0A> {=0A> struct my_file *f =3D v;=0A> size_t rlen;=0A> =0A> /*= Indicate EOF for reads past EOF. */=0A> if (f->offset >=3D 10)=0A> retur= n (0);=0A> =0A> rlen =3D MIN(len, 10 - f->offset);=0A> memcpy(buf, "AAAAA= AAAAA", rlen);=0A> f->offset +=3D rlen;=0A> =0A> return ((int)rlen);=0A> = }=0A> =0A> main()=0A> {=0A> struct my_file *cookie;=0A> FILE *f;=0A> char= buf[100];=0A> size_t x;=0A> =0A> cookie =3D calloc(1, sizeof(*cookie));= =0A> f =3D fopencookie(cookie, "rb", { .read =3D my_read, });=0A> =0A> x = =3D fread(buf, 1, sizeof(buf), f);=0A> ...=0A> }=0A> =0A> Conrad=0A> =0A>= On Thu, Aug 22, 2019 at 9:24 AM Farhan Khan via freebsd-hackers=0A> wrote:=0A> =0A=0AConrad,=0A=0ATook me a while t= o massage that into the format I wanted, but I got it working. Thanks!=0A= =0ATo be honest, I had trouble understanding the manual. I was mistakenly= under the impression that the readfn function was literally just a pass-= through to whatever read mechanism you implement. It also was not obvious= how the return value is interpreted. Do you think it might be useful to = get an update to the documentation? Or was this me failing to understand = the way funopen worked?=0A=0AAlso, Linux's fopencookie(3) has a code samp= le. That might be good to have as well.=0A=0AThanks=0A=0A---=0AFarhan Kha= n=0APGP Fingerprint: 1312 89CE 663E 1EB2 179C 1C83 C41D 2281 F8DA C0DE From owner-freebsd-hackers@freebsd.org Mon Aug 26 01:32:33 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 33975CB4F7 for ; Mon, 26 Aug 2019 01:32:33 +0000 (UTC) (envelope-from rank1seeker@gmail.com) Received: from mail-wr1-x442.google.com (mail-wr1-x442.google.com [IPv6:2a00:1450:4864:20::442]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46GvdD2Xxjz3Ddt for ; Mon, 26 Aug 2019 01:32:32 +0000 (UTC) (envelope-from rank1seeker@gmail.com) Received: by mail-wr1-x442.google.com with SMTP id t16so13675851wra.6 for ; Sun, 25 Aug 2019 18:32:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yg5Ae5dD9OYQfPzLJexXhmsfWaF6CqY2r5UOKPETho0=; b=Hzzdx2em0da06CqUftSjVFcQBFZYY3BhYKh3oYvO0VYXWgE7LRly3u7SBk5ZCAkxvb XP+T5As3m4YNcBbJTHWJXBBDyV6hHUe58iNTNNNLrp3KyRdukZMwcFPScDfpFJD9VoJ6 Zn23Th4KBHW6a61L7BXa3o7ANSqqlwx+zCokWLPd2VvWNPo0/duRtDj+RUt8a1cfasCm kxxjGCxIqs6jl5RcDrDVmyRCf/Puml0LmS0L7F3ni5baBSeGlYlwy053LRtkhxAOE4FO gm0bT3w2POz8rCTvjMoJLl6/yc1XkNITsY8507mXS2UcfuPg/lYHv9m38Pfl8lMJvTkR 6W3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yg5Ae5dD9OYQfPzLJexXhmsfWaF6CqY2r5UOKPETho0=; b=OHhDa44Rec3Lb6TBfW2YZe52o2MRlbXmqQgZHYHJR+4mRTQ11I5KWcUwKgwZcRwGr7 iAW0ycVvgt1SZlc1uCI8eNWsoeBLYJRMGx08aHUYzo6ZEm87Db4qietQEt3ucoLLiTXo Mg6zh6DEYhlOhX/VRjVbQxIQSNusEnLrXk2l9G8/aZOtmCrkgxekkDmydMRPJf19zDZ3 eHLAZF2SraRA1pyqH40njLphCAd6grE5jxorW1ZymU+Gq1drCFm2ExME0dAnlzxn9FcI 1Lg+hqu62Aza1RAaOhuUzvtfUnOJhFGHPH2rkGS+H2YCPyt7d1o5pMqMD4T6QgBkkybS Gctw== X-Gm-Message-State: APjAAAX80XiqdlPK1AEKMLfZPzxfp9SJDKm2Z3eg04u1Z3twVcbup8+2 6wKhAYw88aHpiEYhDbFm6q+mXY0g X-Google-Smtp-Source: APXvYqzDOsr9YHRCHRErsxZ29TKSPnEQz8p/apvc1WrAQfbDCvzYXJKbECaAtcxDUE8FxR9Sa+0dGA== X-Received: by 2002:adf:fe08:: with SMTP id n8mr18344497wrr.60.1566783150395; Sun, 25 Aug 2019 18:32:30 -0700 (PDT) Received: from localhost ([188.252.244.188]) by smtp.gmail.com with ESMTPSA id c8sm11103886wrn.50.2019.08.25.18.32.29 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 25 Aug 2019 18:32:30 -0700 (PDT) Date: Mon, 26 Aug 2019 03:31:17 +0200 From: Domagoj =?UTF-8?Q?Smol=C4=8Di=C4=87?= To: freebsd-hackers@freebsd.org Cc: Jean-Pierre =?UTF-8?Q?Andr=C3=A9?= , freebsd@dussan.org, ntfs-3g-devel@lists.sf.net Subject: Re: mkntfs doesn't install NTFS's bootcode during formatting Message-ID: <20190826033117.000003dc@gmail.com> In-Reply-To: <20190712144429.00000776@gmail.com> References: <20190710192048.00001ca2@gmail.com> <20190712144429.00000776@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 46GvdD2Xxjz3Ddt X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=Hzzdx2em; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rank1seeker@gmail.com designates 2a00:1450:4864:20::442 as permitted sender) smtp.mailfrom=rank1seeker@gmail.com X-Spamd-Result: default: False [-3.98 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.99)[-0.985,0]; RECEIVED_SPAMHAUS_PBL(0.00)[188.244.252.188.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2.4.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(0.00)[ip: (2.67), ipnet: 2a00:1450::/32(-3.00), asn: 15169(-2.33), country: US(-0.05)]; FREEMAIL_CC(0.00)[wanadoo.fr]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Aug 2019 01:32:33 -0000 On Fri, 12 Jul 2019 14:44:29 +0200 Domagoj Smol=C4=8Di=C4=87 wrote: > On Thu, 11 Jul 2019 12:11:32 +0200 > Jean-Pierre Andr=C3=A9 wrote: >=20 > > Domagoj Smol=C4=8Di=C4=87 wrote: =20 > > > FreeBSD 11.2-RELEASE-p9 > > > fusefs-ntfs-2017.3.23 > > > > > > When slice/partition is being formatted, bootcode('; =20 > > containsMicrosoft Windows XP/VISTA bootloader BOOTMGR') isn't > > applied, thus rendering it unbootable. =20 > > > Under Win, after 'bootsect /nt60 ...' has been used on NTFS > > > created =20 > > with mkntfs, THEN it becomes bootable. =20 >=20 > > This BOOTMGR bootcode is proprietary, and it depends on the > > targeted Windows version, hence mkntfs cannot insert it. > > However mkntfs creates the boot sector of the partition, which > > is part of the ntfs file system structure. =20 >=20 > I believe you talk about volume boot record (VBR) being installed. > Yes, but it doesn't install it's part: volume boot code(VBC) whic > targets BOOTMGR(stage 2) >=20 > > Nevertheless can you explain in what circumstances this would > > be useful. On several occasions, I have formatted an ntfs > > partition before installing Window 7 or Windows 10, and these > > installers insert the boot code they want without formatting > > the partition. I do not have XP or Vista any more, but I > > would be very surprised if they do not insert their own boot > > code as well (possibly while reformatting the partition). =20 >=20 > Installers are out of scope here. > Goal is to start installer itself, residing on NTFS > I'm talking about lowest level of boot procedure, JUST after BIOS > "hits" NTFS. >=20 > > > > > > When slice/partition is being formatted directly under Win, > > > bootcode =20 > > IS also being applied. =20 >=20 > > No, this is not done while formatting. The Windows formatting > > is limited, and the full formatting occurs when the partition > > is mounted the first time. So, when upgrading Windows, you can > > format with the old version before installing the new one. =20 >=20 > You are wrong. I've tested it's presence with hex. > After mkntfs, I copy files(install procedure for Win7) and device > won't boot. It does after after 'bootsect /nt60 ...' is executed > under Win7. (anoying part) >=20 > If I do ONLY formating under Win7, then plug it back into FreeBSD, > copy mentioned files and boot decvice, Win7 installation starts. >=20 > > > So, if you need a bootable NTFS, why to bother in a first place > > > with =20 > > mkntfs, then transferring device to Win machine and using command > > line under Win to run 'bootsect' tool, when you can simply click > > 'Format...' & Start?!? > > Can you explain what you want to boot into ? > >=20 > > Jean-Pierre =20 >=20 >=20 > I want to use ONLY FreeBSD to create custom bootable Win7 install > media. Only obstacle left is NTFS's stage 1 bootcode =3D> VBC. >=20 > I believe I've located it's position and size in slice (7k or 8k, > can't remember) So I'll attempt to extract it with dd tool and later > apply it just after mkntfs. But then again, I believe (searching at a > first glance) that this functionality exited and was removed because > of a "proprietary issues". So versions older than > ntfs-3g-2016.2.22-2, should be used. I believe it was about > "ntfsprogs/boot.c". Can anyone confirm this? >=20 > How about updating FreeBSD's license by keeping it BSD, but adding > EXCEPTIONS for Microsoft and other big corporations with same > "proprietary issues". In life, it is important to be equal in justice > to each other. If they claim it's ok and normal, just do the same to > them. If they complain ... everything is clear. ;) >=20 >=20 > Domagoj Smol=C4=8Di=C4=87 So ... I wondered why changelog doesn't have to say anything about "feature" removal, or to better say a degradation? https://www.tuxera.com/community/release-history/ Official note on 'STABLE Version 2016.2.22 (March 21, 2016)' is the last on= e listing 'Changes to mkntfs' and it says NOTHING about functionality removal. What is even worse, man pages are left to be misleading and here I mean on = mkntfs's flags: -p, -S & -H, which all and mkntfs itslef warns that NTFS won't be bootabl= e, if not, or improperly set! I've lost too much time "hanging" here, thinking that I'm blind and don't s= ee obvious! Somwhere "else", Jean-Pierre has pointed out that UNdocumented change has b= een done to boot.c file. And to use v2016.2.22 or below in order to be able to create bootable NTFS = with mkntfs. No one is going to convince me, that this uninforming act is simply acciden= tal! This is sneaky and subtle manipulation and reasons should be pretty much cl= ear to everyone. Well ..., let's take it into community's advantage ... =46rom legal point of view, if something isn't stated, it doesn't exist and d= oesn't apply, therefore, can't provoke any legal act. So ..., let's make NTFS-3G into accordance with it's official and currently= effective release history AND man pages! # cd /usr/ports/sysutils/fusefs-ntfs # make config Maintainer should add here another option to chose from, named: FETCH DA PATCH Hell it even rhymes! :D And it would do: # make extract # fetch -qo - 'https://download.tuxera.com/opensource/ntfs-3g_ntfsprogs-201= 6.2.22.tgz' | tar -xf - -q --include=3Dntfs-3g_ntfsprogs-2016.2.22/ntfsprog= s/boot.c --strip-components 2 -C work/ntfs-3g_ntfsprogs-2017.3.23/ntfsprogs # make install clean TEST: Created 2 NTFSs with mkntfs, with only -p flag specified (-S & -H wer= en't necessary). First on partition to boot win7's setup and second on partition of win7's f= resh installation. They BOTH booted! Voila! Resulting installation (functionality in accordance with official docs and = mans): ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Origin: sysutils/fusefs-ntfs Name: fusefs-ntfs-2017.3.23 Mount NTFS partitions (read/write) and disk images Architecture: FreeBSD:11:i386 Maintainer's email: freebsd@dussan.org Home page: https://www.tuxera.com/community/open-source-ntfs-3g/ Installed at: 24-08(August)-2019 17h:11m:30s (UTC) License(s): GPLv2+ On install: NTFS-3G has been installed, for information, known issues and how to report bugs see the FreeBSD README: /usr/local/share/doc/ntfs-3g/README.FreeBSD Also see the official README (but has some Linux specific parts). Listing fusefs-ntfs-2017.3.23's installed files: Licence's dir: /usr/local/share/licenses/fusefs-ntfs-2017.3.23/ Shared objects: /usr/local/lib/libntfs-3g.so /usr/local/lib/libntfs-3g.so.88 /usr/local/lib/libntfs-3g.so.88.0.0 (s)bin: /usr/local/bin/lowntfs-3g /usr/local/bin/ntfs-3g /usr/local/bin/ntfs-3g.probe /usr/local/bin/ntfscat /usr/local/bin/ntfscluster /usr/local/bin/ntfscmp /usr/local/bin/ntfsfix /usr/local/bin/ntfsinfo /usr/local/bin/ntfsls /usr/local/sbin/mkntfs /usr/local/sbin/ntfsclone /usr/local/sbin/ntfscp /usr/local/sbin/ntfslabel /usr/local/sbin/ntfsrecover /usr/local/sbin/ntfsresize /usr/local/sbin/ntfsundelete man: (Section 3 excluded) # man 8 mkntfs # man 8 ntfs-3g # man 8 ntfs-3g.probe # man 8 ntfscat # man 8 ntfsclone # man 8 ntfscluster # man 8 ntfscmp # man 8 ntfscp # man 8 ntfsdecrypt # man 8 ntfsfallocate # man 8 ntfsfix # man 8 ntfsinfo # man 8 ntfslabel # man 8 ntfsls # man 8 ntfsprogs # man 8 ntfsrecover # man 8 ntfsresize # man 8 ntfssecaudit # man 8 ntfstruncate # man 8 ntfsundelete # man 8 ntfsusermap # man 8 ntfswipe Space consumption: 1.58MiB The ntfs-3g driver is an open source, freely available read/write NTFS driver, which provides safe and fast handling of the Windows XP, Windows Server 2003, Windows 2000, Windows Vista, Windows Server 2008, Windows 7 and Windows 8 NTFS file systems. Almost the full POSIX filesystem functionality is supported, the major exceptions are changing the file ownerships and the access rights. WWW: https://www.tuxera.com/community/open-source-ntfs-3g/ Other files: /usr/local/lib/libntfs-3g.a /usr/local/share/doc/ntfs-3g/README /usr/local/share/doc/ntfs-3g/README.FreeBSD ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Have a fun crew! ;) Domagoj Smol=C4=8Di=C4=87 From owner-freebsd-hackers@freebsd.org Mon Aug 26 16:56:30 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3CAFADFFD4 for ; Mon, 26 Aug 2019 16:56:30 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 46HJ7K6lMLz4dyP for ; Mon, 26 Aug 2019 16:56:29 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id E44A6DFFD2; Mon, 26 Aug 2019 16:56:29 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E3B33DFFD0; Mon, 26 Aug 2019 16:56:29 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-io1-f44.google.com (mail-io1-f44.google.com [209.85.166.44]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46HJ7K0c6Fz4dyM; Mon, 26 Aug 2019 16:56:28 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-io1-f44.google.com with SMTP id z3so39033573iog.0; Mon, 26 Aug 2019 09:56:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:content-transfer-encoding; bh=VanxT31TJS4nyEq2cRp13k5h7f6TmBbfy+MzqPZxmHQ=; b=LzCz8ku6ZU2t0/YROLGh3ykP41AAZecFN5R/OTHNnNLwh3sY4Szv/hRliEBkeF09zr 59+wJmNfTlaK0uu+l3tgg4rLjRXJId9gA9NpNcT+G/Lg337xTpmJVMbJzcZwEZU1OwyT MJYKlCyvFVXOt31urCKbUsV5OldbAIilASk7MoEhl5eHww9cDJ1dbMoGm6uETbz77vCO /iJuGZp1uQH2mvtCE6r0o7ymeZtGsqMEIbw38u10sjV/4s+mWRuQR8lEzKevx6mlIbv3 6pEycXSDbU05zYc2LjeWBjY3FSsLFoFyHMT9Hsra1l3YPVjY3OwLJtgKNZxNlV5FIlHi H16g== X-Gm-Message-State: APjAAAWZ4CvFnck0l7EgvVPDUzJUw3vU2vn1uCEOTtQNhtJcyUB8j8tz kMMj2jTf4DzcJpvFt+xJZaMctTwqnV4zOFI0YSHzN4a5 X-Google-Smtp-Source: APXvYqz5U9xXJSCKmRARMhTKS+fsQpNeZppY2a/xFa0pH0JPJIPxtqJBzECPNQnYAMfFqGksZS0GZsJuwglHlQtElG0= X-Received: by 2002:a6b:ea16:: with SMTP id m22mr24349319ioc.115.1566838586619; Mon, 26 Aug 2019 09:56:26 -0700 (PDT) MIME-Version: 1.0 References: <20190825122332.GA16293@v2> In-Reply-To: <20190825122332.GA16293@v2> From: Ed Maste Date: Mon, 26 Aug 2019 12:56:07 -0400 Message-ID: Subject: Re: FreeBSD Quarterly Status Report - Second Quarter 2019 To: hackers@freebsd.org, current , stable@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 46HJ7K0c6Fz4dyM X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.44 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-5.02 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[freebsd.org]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE(-2.03)[ip: (-4.41), ipnet: 209.85.128.0/17(-3.35), asn: 15169(-2.33), country: US(-0.05)]; NEURAL_HAM_SHORT(-0.99)[-0.992,0]; RCVD_IN_DNSWL_NONE(0.00)[44.166.85.209.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Aug 2019 16:56:30 -0000 On Sun, 25 Aug 2019 at 08:24, Edward Tomasz Napiera=C5=82a wrote: > > FreeBSD Project Quarterly Status Report - 2nd Quarter 2019 Due to an oversight the FreeBSD Foundation's quartery status entry was omitted from this report. Contact: Deb Goodkin The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Funding comes from individual and corporate donations and is used to fund and manage software development projects, conferences and developer summits, and provide travel grants to FreeBSD contributors. The Foundation purchases and supports hardware to improve and maintain FreeBSD infrastructure and provides resources to improve security, quality assurance, and release engineering efforts; publishes marketing material to promote, educate, and advocate for the FreeBSD Project; facilitates collaboration between commercial vendors and FreeBSD developers; and finally, represents the FreeBSD Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity. Here are some highlights of what we did to help FreeBSD last quarter: We held our annual board meeting in Ottawa on May 14. Board Director and Officer elections take place each year at this meeting. Justin Gibbs was elected as the new President of the Board of Directors. The new FreeBSD Foundation Board of Directors includes President and Founder Justin T. Gibbs, Vice President Benedict Reuschling, Secretary Philip Paeps, Treasurer Marshall Kirk McKusick, and Directors Hiroki Sato, George Neville-Neil and Robert N. M. Watson. You can read more about the elections at: . After the elections, our management team gave updates to the board on their respective areas. We then discussed the key areas of the Project that need help, and where we can step in to fill those holes. We reviewed and updated our 12 month goals, and identified projects we should support. We then discussed conferences we are likely to attend, and went over the latest on our fundraising efforts. We followed that up with a discussion on how to get more users to contribute back to the Project. While discussing how to increase the number of users and contributors, we talked about methods for making for more training material available. Partnerships and Commercial User Support We help facilitate collaboration between commercial users and FreeBSD developers. We also meet with companies to discuss their needs and bring that information back to the Project. In Q2, Ed Maste and Deb Goodkin met with a few commercial users in Germany. It=E2=80=99s not only beneficial for the above, but it also helps us understand some of the applications where FreeBSD is used. Because BSDCan brings in a high number of commercial users, we have an excellent opportunity to have similar discussions about their needs during the four-day FreeBSD Summit and BSDCan. Fundraising Efforts Our work is 100% funded by your donations. We are grateful for the generous donations from Intel, NetApp, VMware and Stormshield last quarter. We are working hard to get more commercial users to give back to help us continue our work supporting FreeBSD. More importantly, we=E2=80=99d like to thank our individual donors, for making $10-$1,000 donations last quarter, for a total of $16,000! Please consider making a donation to help us continue and increase our support for FreeBSD: https://www.FreeBSDfoundation.org/donate/. We also have the Partnership Program, to provide more benefits for our larger commercial donors. Find out more information at https://www.FreeBSDfoundation.org/FreeBSD-foundation-partnership-program/ and share with your companies! OS Improvements The Foundation improves the FreeBSD operating system by employing our technical staff to maintain and improve critical kernel subsystems, add features and functionality, and fix problems. The Foundation also provides grants to fund individual projects. There were 243 commits to the FreeBSD base system repository sponsored by the Foundation during the quarter. These include improvements to the tmpfs in-memory, MSDOS, and UFS filesystems, device driver and hardware compatibility fixes, virtual memory (VM), tool chain, documentation, and testing and continuous integration improvements. We fixed a number of race conditions and security issues found by Syzkaller, Google=E2=80=99s code-coverage-guided system call fuzzer. Alan Somers=E2=80=99 work on updating FreeBSD=E2=80=99s support for FUSE (u= serspace filesystems) continued during the quarter; the full details are elsewhere in this quarterly report. At this point most of the work has been committed to the project branch but some bug fixes and improvements have been committed directly to the FreeBSD development branch. Edward Napierala=E2=80=99s Linuxulator project continued through the quarte= r, resulting in a number of improvements to the Linuxulator and linux-specific functionality such as linsysfs. This work is part of the path to supporting the Linux strace debugging tool in order to facilitate debugging failures of other Linux binaries under the Linuxulator. Mateusz Guzik continued with scalability and performance improvements during the quarter, and Bjoern Zeeb integrated the SDIO stack (with details elsewhere in the quarterly report). Progress was made on the online RAID-Z expansion project over the quarter. Matt Ahrens posted an alpha preview of the feature for further experimentation and review, and the FreeBSD Foundation will make an alpha release image available for testing in the near future. Foundation staff contributed to nine FreeBSD security advisories and errata updates over the quarter, including CPU vulnerability workarounds. Related work included improving Intel microcode update loading. Continuous Integration and Quality Assurance The Foundation provides a full-time staff member who is working on improving our automated testing, continuous integration, and overall quality assurance efforts. During the second quarter of 2019, Foundation staff continued to improve the project's CI infrastructure, worked with contributors to fix the failing build and test cases, and worked with other teams in the Project for their testing needs. We hosted a CI-focused working group at BSDcan and continue to publish the CI weekly report at freebsd-testing@ mailing list. See the FreeBSD CI section of this report for more information. Supporting FreeBSD Infrastructure The Foundation provides hardware and support to improve the FreeBSD infrastructure. Last quarter, we continued supporting FreeBSD hardware located around the world. FreeBSD Advocacy and Education A large part of our efforts are dedicated to advocating for the Project. This includes promoting work being done by others with FreeBSD; producing advocacy literature to teach people about FreeBSD and help make the path to starting using FreeBSD or contributing to the Project easier; and attending and getting other FreeBSD contributors to volunteer to run FreeBSD events, staff FreeBSD tables, and give FreeBSD presentations. The FreeBSD Foundation sponsors many conferences, events, and summits around the globe. These events can be BSD-related, open source, or technology events geared towards underrepresented groups. We support the FreeBSD-focused events to help provide a venue for sharing knowledge, to work together on projects, and to facilitate collaboration between developers and commercial users. This all helps provide a healthy ecosystem. We support the non-FreeBSD events to promote and raise awareness of FreeBSD, to increase the use of FreeBSD in different applications, and to recruit more contributors to the Project. Check out some of the advocacy and education work we did last quarter: - Represented FreeBSD at LinuxFest Northwest In Bellingham, Washington - Sponsored and helped organize the FreeBSD Developers Summit at BSDCan, in Ottawa, Canada - Sponsored and attended BSDCan 2019 - Set up registration and attended the Vienna FreeBSD Security Hackathon in Vienna, Austria - Represented FreeBSD at HKOSCON - Attended the Berlin FreeBSD Developers Summit - Presented at 2019 Comcast Labs Connect Open Source Conference Sponsored, presented and represented FreeBSD at RootConf 2019 in Bangalore, India - Committed to attend OSCON, and All Things Open - Committed to sponsor and help organize a Bay Area Developers Summit - Provided FreeBSD advocacy material - Provided travel grants to FreeBSD contributors to attend many of the above events We continued producing FreeBSD advocacy material to help people promote FreeBSD around the world. Read more about our conference adventures in the conference recaps and trip reports in our monthly newsletters: https://www.freebsdfoundation.org/news-and-events/newsletter/ We help educate the world about FreeBSD by publishing the professionally produced FreeBSD Journal. As we mentioned previously, the FreeBSD Journal is now a free publication. Find out more and access the latest issues at https://www.FreeBSDfoundation.org/journal/. You can find out more about events we attended and upcoming events at https://www.FreeBSDfoundation.org/news-and-events/. We have continued our work with a new website developer to help us improve our website. Work has begun to make it easier for community members to find information more easily and to make the site more efficient. Legal/FreeBSD IP The Foundation owns the FreeBSD trademarks, and it is our responsibility to protect them. We also provide legal support for the core team to investigate questions that arise. Go to http://www.FreeBSDfoundation.org to find out how we support FreeBSD and how we can help you! From owner-freebsd-hackers@freebsd.org Wed Aug 28 04:30:16 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B7A93D0D11; Wed, 28 Aug 2019 04:30:16 +0000 (UTC) (envelope-from lwhsu.freebsd@gmail.com) Received: from mail-yb1-f172.google.com (mail-yb1-f172.google.com [209.85.219.172]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46JCTM36jtz3DlG; Wed, 28 Aug 2019 04:30:12 +0000 (UTC) (envelope-from lwhsu.freebsd@gmail.com) Received: by mail-yb1-f172.google.com with SMTP id z2so355118ybp.9; Tue, 27 Aug 2019 21:30:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=QYAnd6yKa5/PNe/LsQCvO+Q0ux1DvZfWAhvQc/CmPD8=; b=H3RApEg1RsOEC5KBy0eZABl+zjDWvmicgNmJXhBWPkpK0ww8iOe9pxbsRQFQeSPJBP qJkZR4Rv27dRNabZtsWYEeVIVlF5j7Lq3RloPvM31SfO/RMfpj4iJa+NU8kzlCtH2NLX xkcHpZaCMuRAgwAMK9weSZeperI9nS92PgUCrLJbmXJ6JHpYeMfRPqSUqGXgoXhHtNvL /VIw5G+QjxbQAvM2xeSiSvtp9oTiv3z9ZU+WkcD7Ln56ioD0SYVA5fe/WqsBmmVs6A6N BTHUVCrdCsXh8YQnfWwufFzjc7pRoxvM2oHwTBmFq3iMbjkWkGPJHHIhYPWP6+GppLrV dnEQ== X-Gm-Message-State: APjAAAXoVoK5LNPEXpQI3X7YJzywDvqEA4q4oDl4fWcJXErCkOMHy47Y osno0WiT20TAWDvFuDb4D18fxbn7J+TsI+zDmVcGSVaS X-Google-Smtp-Source: APXvYqwosnRMDk0Oi+gCXcuRruoHUyL4Wp5xhPCxgQyHBuBxpiubCemNl1fnMU7Q5LDnwH0dWZrA0Q5zIz3VfUS8+/8= X-Received: by 2002:a25:7089:: with SMTP id l131mr1629577ybc.110.1566966610613; Tue, 27 Aug 2019 21:30:10 -0700 (PDT) MIME-Version: 1.0 From: Li-Wen Hsu Date: Wed, 28 Aug 2019 12:29:58 +0800 Message-ID: Subject: FCP 20190401-ci_policy: CI policy To: fcp@freebsd.org, FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46JCTM36jtz3DlG X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of lwhsufreebsd@gmail.com designates 209.85.219.172 as permitted sender) smtp.mailfrom=lwhsufreebsd@gmail.com X-Spamd-Result: default: False [-5.98 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE(-2.98)[ip: (-9.20), ipnet: 209.85.128.0/17(-3.34), asn: 15169(-2.33), country: US(-0.05)]; NEURAL_HAM_SHORT(-0.99)[-0.993,0]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[172.219.85.209.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[lwhsu@freebsd.org,lwhsufreebsd@gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; TAGGED_FROM(0.00)[]; FROM_NEQ_ENVFROM(0.00)[lwhsu@freebsd.org,lwhsufreebsd@gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Aug 2019 04:30:16 -0000 It seems I was doing wrong that just changed the content of this FCP to "feedback", but did not send to the right mailing lists. So I would like to make an announcement that the FCP 20190401-ci_policy "CI policy": https://github.com/freebsd/fcp/blob/master/fcp-20190401-ci_policy.md is officially in "feedback" state to hopefully receive more comments and suggestions, then we can move on for the next FCP state. Thanks, Li-Wen From owner-freebsd-hackers@freebsd.org Wed Aug 28 09:07:46 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 82D99D6800 for ; Wed, 28 Aug 2019 09:07:46 +0000 (UTC) (envelope-from yuripv@yuripv.net) Received: from new4-smtp.messagingengine.com (new4-smtp.messagingengine.com [66.111.4.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46JKdY52H7z3xmd for ; Wed, 28 Aug 2019 09:07:45 +0000 (UTC) (envelope-from yuripv@yuripv.net) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailnew.nyi.internal (Postfix) with ESMTP id C64432566 for ; Wed, 28 Aug 2019 05:07:44 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Wed, 28 Aug 2019 05:07:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yuripv.net; h=to :from:subject:message-id:date:mime-version:content-type :content-transfer-encoding; s=fm1; bh=y0Gci8I5+yZTQKRME9Rvjdx4/j 7M38193i63JTtPRNc=; b=ZFPMuem31xU+uRxf1j4UeVreDolyuZ/JSpv/2o5dCr C2pbHZlIVODvciCN/LLj4pLb15SYjN878+j8wCITwCFfFtLsD6RW0eX1ZtoAF6ZA cVvH6vk01TM8+J5azIYVsHJsd22VxGiUIyxy87b9mQpdF0p3xgNUeqgiAlL+5BdF XAy093Ormqss2V6vzbVeVnxE9fZjxbwv9MwlFmBo/dAkzQOs+FLAcMw2QKcggdyr fgaLFYd+q9Zyb2TF8NY2oRLZmhF1g++Xg6/Y+TgWGRLxh17C0zgWM/O2N74/gnhK X5vT77brflXiWARoK4sHuzWzAtGM89fLOF45eM5k3LnQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=y0Gci8 I5+yZTQKRME9Rvjdx4/j7M38193i63JTtPRNc=; b=Pdk3ZXh6rA7Y5kWFOZEZ9n HKmSnBicDYItse2lSCvdBnrIaQdjNFEDOJeV+Ctj85vWz3EpU2S6y54blks/Ek2z jk30ue33RFq+z9gOGVoT63umql2eGhYI/l3sG0oCRmQ+dSPHdSoedaXon6BkpIZJ mC1iYgRUFbWxWsZ349N6y0xalsX1hpvPHTKIvyrdM3cWt+Q4tLAWtd2gX1edJOsV 0OgKNTv/yYFWm40U+ES1EuhbRNGtCqHzUQAmykUw5EUjnfd//8auCo8A8xJeycoz 6rs9IYH4NgXyJDWLMyjzZaEupaFKPVnzfDmQcGx7V/hYQEdSGtpRkDGnR39pH9AA == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrudeitddguddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefvhffukffffgggtgfgsehtjeertd dtfeejnecuhfhrohhmpegjuhhrihcurfgrnhhkohhvuceohihurhhiphhvseihuhhrihhp vhdrnhgvtheqnecukfhppeduleehrddvtdeirddukeefrdduieejnecurfgrrhgrmhepmh grihhlfhhrohhmpeihuhhrihhpvheshihurhhiphhvrdhnvghtnecuvehluhhsthgvrhfu ihiivgeptd X-ME-Proxy: Received: from [192.168.1.2] (unknown [195.206.183.167]) by mail.messagingengine.com (Postfix) with ESMTPA id DE30680064 for ; Wed, 28 Aug 2019 05:07:43 -0400 (EDT) To: freebsd-hackers@freebsd.org From: Yuri Pankov Subject: ichsmb(4) and msleep() Message-ID: <7dfebbd3-85d6-c7b7-b83b-fae8b644649e@yuripv.net> Date: Wed, 28 Aug 2019 12:07:42 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46JKdY52H7z3xmd X-Spamd-Bar: ------- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yuripv.net header.s=fm1 header.b=ZFPMuem3; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=Pdk3ZXh6; dmarc=none; spf=pass (mx1.freebsd.org: domain of yuripv@yuripv.net designates 66.111.4.230 as permitted sender) smtp.mailfrom=yuripv@yuripv.net X-Spamd-Result: default: False [-7.08 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[yuripv.net:s=fm1,messagingengine.com:s=fm3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.230]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[yuripv.net:+,messagingengine.com:+]; NEURAL_HAM_SHORT(-0.99)[-0.992,0]; DMARC_NA(0.00)[yuripv.net]; IP_SCORE(-3.49)[ip: (-9.87), ipnet: 66.111.4.0/24(-4.84), asn: 11403(-2.68), country: US(-0.05)]; RCVD_IN_DNSWL_LOW(-0.10)[230.4.111.66.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Aug 2019 09:07:46 -0000 I have a "timed sleep before timers are working" panic in ichsmb_readb() calling ichsmb_wait() which uses msleep(). That is trying to jedec_dimm(4) module so it's trying to attach pretty early in boot. What would be the correct replacement for msleep() here? From owner-freebsd-hackers@freebsd.org Wed Aug 28 09:27:43 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DCE27D6F37 for ; Wed, 28 Aug 2019 09:27:43 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [88.99.82.50]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46JL4Z4Q7fz3yp9 for ; Wed, 28 Aug 2019 09:27:42 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.129.235]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 10DD8260240; Wed, 28 Aug 2019 11:27:39 +0200 (CEST) Subject: Re: ichsmb(4) and msleep() To: Yuri Pankov , freebsd-hackers@freebsd.org References: <7dfebbd3-85d6-c7b7-b83b-fae8b644649e@yuripv.net> From: Hans Petter Selasky Message-ID: <478965aa-5256-e356-5339-de6fb82c3459@selasky.org> Date: Wed, 28 Aug 2019 11:26:56 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <7dfebbd3-85d6-c7b7-b83b-fae8b644649e@yuripv.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46JL4Z4Q7fz3yp9 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of hps@selasky.org designates 88.99.82.50 as permitted sender) smtp.mailfrom=hps@selasky.org X-Spamd-Result: default: False [-6.47 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:mail.turbocat.net:c]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[selasky.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.98)[-0.984,0]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-3.19)[ip: (-9.37), ipnet: 88.99.0.0/16(-4.75), asn: 24940(-1.80), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:88.99.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Aug 2019 09:27:43 -0000 On 2019-08-28 11:07, Yuri Pankov wrote: > I have a "timed sleep before timers are working" panic in ichsmb_readb() > calling ichsmb_wait() which uses msleep(). That is trying to > jedec_dimm(4) module so it's trying to attach pretty early in boot. > What would be the correct replacement for msleep() here? > If you only need a sleep-delay, pause() is the right one. It handles cold-boot. --HPS From owner-freebsd-hackers@freebsd.org Wed Aug 28 09:44:18 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5E00DD757B for ; Wed, 28 Aug 2019 09:44:18 +0000 (UTC) (envelope-from yuripv@yuripv.net) Received: from new4-smtp.messagingengine.com (new4-smtp.messagingengine.com [66.111.4.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46JLRj3yYBz40hj for ; Wed, 28 Aug 2019 09:44:16 +0000 (UTC) (envelope-from yuripv@yuripv.net) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailnew.nyi.internal (Postfix) with ESMTP id CE2D83C1C; Wed, 28 Aug 2019 05:44:15 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Wed, 28 Aug 2019 05:44:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yuripv.net; h= subject:to:references:from:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=fm1; bh=X BYPAxu5G0TLzdfGrQEBfes7cR0krwwcMpb5FUHdW7Y=; b=vM6P4w5gdzDYQikte Vsd/U5mazjrC8qN3jJD0hSnwLGs2w37CJAIoG/3ZYLC44x8J4SozEi5sOWTc/EI2 +VNmedjMkCTiW1D68muaWgttDXfjpSE7KteqVtG8RBtLSpOZ3tKmN5YyE7qJ3Zj4 rLjFxHN7Y347XfUe/V2206mLMk06zlR5tQHUpmgKEar8n5jKW6sWgKX/jhIbZ2Ko jpaQ8ncjwlejKSCmkJwmeylsF9YVBhzwZIn2gXzvd8s/znGxzMHZJQb0XXBKTpet tmvKmrg6Y7bUTxR66phMCzSWmeZrFfK/fExACCJ2VO1D1XlNAhWYFMDWguvThdg4 FPvyQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=XBYPAxu5G0TLzdfGrQEBfes7cR0krwwcMpb5FUHdW 7Y=; b=hJ58CQP7G3OF6Gd0T4Ps0XADSAoAgjxB9HFE6XIUmdmNSQZMUsTkmk8nB eiYhWXnHEnyv8EGh9CLOfMDTBF4NEk3+5+Fk3WsEWeLq6FjxhdVg8xtebTS+GNr+ tGp5F+8ozGlzR5hQGxeozGJnAnH0GHne+6hE1I5usjY9z/VLoH0pKcgFjw4JD5Ll QZJQpbdh9yL8xenUDhzv2J+tdiJR7YPE7Qr+afgTbZ0z02NXWzVEPvjCSIB/VRfp FIZAsn/Pf57yDcNbhwM345p+ZDWMepxaQ0JayVIgP4kOGrFdxmja49+SF8R/PJ+V RvPDYnfCYVi9wVu2Ni21V3fml4E8g== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrudeitddgudelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefuvfhfhffkffgfgggjtgfgsehtje ertddtfeejnecuhfhrohhmpegjuhhrihcurfgrnhhkohhvuceohihurhhiphhvseihuhhr ihhpvhdrnhgvtheqnecukfhppeduleehrddvtdeirddukeefrdduieejnecurfgrrhgrmh epmhgrihhlfhhrohhmpeihuhhrihhpvheshihurhhiphhvrdhnvghtnecuvehluhhsthgv rhfuihiivgeptd X-ME-Proxy: Received: from [192.168.1.2] (unknown [195.206.183.167]) by mail.messagingengine.com (Postfix) with ESMTPA id A38A980059; Wed, 28 Aug 2019 05:44:14 -0400 (EDT) Subject: Re: ichsmb(4) and msleep() To: Hans Petter Selasky , freebsd-hackers@freebsd.org References: <7dfebbd3-85d6-c7b7-b83b-fae8b644649e@yuripv.net> <478965aa-5256-e356-5339-de6fb82c3459@selasky.org> From: Yuri Pankov Message-ID: <63daa36a-5c22-6b08-3cd7-562fa961ab61@yuripv.net> Date: Wed, 28 Aug 2019 12:44:13 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <478965aa-5256-e356-5339-de6fb82c3459@selasky.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46JLRj3yYBz40hj X-Spamd-Bar: ------- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yuripv.net header.s=fm1 header.b=vM6P4w5g; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=hJ58CQP7; dmarc=none; spf=pass (mx1.freebsd.org: domain of yuripv@yuripv.net designates 66.111.4.230 as permitted sender) smtp.mailfrom=yuripv@yuripv.net X-Spamd-Result: default: False [-7.08 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[yuripv.net:s=fm1,messagingengine.com:s=fm3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.230:c]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[yuripv.net]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[yuripv.net:+,messagingengine.com:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.99)[-0.991,0]; IP_SCORE(-3.49)[ip: (-9.87), ipnet: 66.111.4.0/24(-4.84), asn: 11403(-2.68), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[230.4.111.66.list.dnswl.org : 127.0.5.1] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Aug 2019 09:44:18 -0000 Hans Petter Selasky wrote: > On 2019-08-28 11:07, Yuri Pankov wrote: >> I have a "timed sleep before timers are working" panic in ichsmb_readb() >> calling ichsmb_wait() which uses msleep(). That is trying to >> jedec_dimm(4) module so it's trying to attach pretty early in boot. >> What would be the correct replacement for msleep() here? >> > > If you only need a sleep-delay, pause() is the right one. It handles > cold-boot. I guess that won't work here as we need to be waked up by interrupt handler on command completion, and pause() seems to sleep unconditionally for the given time in 'cold' case (if I'm reading the code correctly). From owner-freebsd-hackers@freebsd.org Wed Aug 28 09:47:53 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D50D1D76E3 for ; Wed, 28 Aug 2019 09:47:53 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [IPv6:2a01:4f8:c17:6c4b::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46JLWr5my3z40rn for ; Wed, 28 Aug 2019 09:47:52 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.129.235]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 4D064260240; Wed, 28 Aug 2019 11:47:50 +0200 (CEST) Subject: Re: ichsmb(4) and msleep() To: Yuri Pankov , freebsd-hackers@freebsd.org References: <7dfebbd3-85d6-c7b7-b83b-fae8b644649e@yuripv.net> <478965aa-5256-e356-5339-de6fb82c3459@selasky.org> <63daa36a-5c22-6b08-3cd7-562fa961ab61@yuripv.net> From: Hans Petter Selasky Message-ID: <7f6de96d-8b56-e242-8950-04a20b197bce@selasky.org> Date: Wed, 28 Aug 2019 11:47:07 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <63daa36a-5c22-6b08-3cd7-562fa961ab61@yuripv.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46JLWr5my3z40rn X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of hps@selasky.org designates 2a01:4f8:c17:6c4b::2 as permitted sender) smtp.mailfrom=hps@selasky.org X-Spamd-Result: default: False [-5.86 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:mail.turbocat.net:c]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[selasky.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.98)[-0.983,0]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-2.58)[ip: (-9.11), ipnet: 2a01:4f8::/29(-1.97), asn: 24940(-1.80), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Aug 2019 09:47:53 -0000 On 2019-08-28 11:44, Yuri Pankov wrote: > Hans Petter Selasky wrote: >> On 2019-08-28 11:07, Yuri Pankov wrote: >>> I have a "timed sleep before timers are working" panic in ichsmb_readb() >>> calling ichsmb_wait() which uses msleep(). That is trying to >>> jedec_dimm(4) module so it's trying to attach pretty early in boot. >>> What would be the correct replacement for msleep() here? >>> >> >> If you only need a sleep-delay, pause() is the right one. It handles >> cold-boot. > > I guess that won't work here as we need to be waked up by interrupt > handler on command completion, and pause() seems to sleep > unconditionally for the given time in 'cold' case (if I'm reading the > code correctly). If you are too early inside a SYSINIT() path, then you cannot use sleeping. You will have to use polling in a loop with a fixed DELAY() to know the timeout. --HPS From owner-freebsd-hackers@freebsd.org Wed Aug 28 11:41:12 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 70DCAD9C2E for ; Wed, 28 Aug 2019 11:41:12 +0000 (UTC) (envelope-from yuripv@yuripv.net) Received: from new2-smtp.messagingengine.com (new2-smtp.messagingengine.com [66.111.4.224]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46JP2b2BF7z45kl for ; Wed, 28 Aug 2019 11:41:11 +0000 (UTC) (envelope-from yuripv@yuripv.net) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailnew.nyi.internal (Postfix) with ESMTP id 69298A91; Wed, 28 Aug 2019 07:41:10 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Wed, 28 Aug 2019 07:41:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yuripv.net; h= subject:to:references:from:message-id:date:mime-version :in-reply-to:content-type; s=fm1; bh=Nc88nQg0YlS2Cz7A5UE/ZBeZu3p nq0CUkZWENzOzUpc=; b=At1JIRRTtiHor9KCdeknHXFIyvryuWGsnjonflz18dE W8BHso+GdFObHeVzOa27JbAxDUg9SiTxUAUKFhHjQctFlYG5PP2OCpIkvXgepx8/ eXMZzICKj0A7wicw2lzQ4YkzjjgWk7ux3JRoYeOBfVfHSzCGWsyYpoAfrwJCJOJN 2Mk/Vgyqikv/QqURC5tRNVzZ1ZyEu1snew6VIQsNKLUZrkZHsou4+V/Xk7Qek/rJ iyoa04veIY2E0uP1UyX9mBNTreddTJsgVs5Y06clC82Hy9bwvTXTn3t21A41ndhg OXegWtfE09bm64g3n1a4MyfqsUtXdUrtuooDZPMmhOQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=Nc88nQ g0YlS2Cz7A5UE/ZBeZu3pnq0CUkZWENzOzUpc=; b=OYkk18KUpcGje4GYauWy02 LChu+hCvzOl818PAlXvRW3H1XX0vRAID2PI7v20leJRsxx6VYN9B1jadnqu43sCs iBkv465g9s3pnoEjeMNv5HgnlRDBEoGXQali4FfOLRnt9ErKTqiEzJ36xGp6AQQE BwNFCkjgnPAD/+ATkHJ/1G1ov5uwBbxGuDBOGmePYHjutV47go75KUV6fHahAu96 OZE+z3u37Tfv39bZM6ss1ht4L8DFshzyLAYIijHS1BNLBYYgrgLwv1e8RTIztbcs Z8ywtgUnxM3FF+2UqoAzUwvT4JPn/F++c2ko/IBFQj8a0PF1zNh4eiqUzLkN1ocA == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrudeitddggedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefuvfhfhffkffgfgggjtgesmhdtre ertdefjeenucfhrhhomhepjghurhhiucfrrghnkhhovhcuoeihuhhrihhpvheshihurhhi phhvrdhnvghtqeenucfkphepudelhedrvddtiedrudekfedrudeijeenucfrrghrrghmpe hmrghilhhfrhhomhephihurhhiphhvseihuhhrihhpvhdrnhgvthenucevlhhushhtvghr ufhiiigvpedt X-ME-Proxy: Received: from [192.168.1.2] (unknown [195.206.183.167]) by mail.messagingengine.com (Postfix) with ESMTPA id 2573F8005A; Wed, 28 Aug 2019 07:41:09 -0400 (EDT) Subject: Re: ichsmb(4) and msleep() To: Hans Petter Selasky , freebsd-hackers@freebsd.org References: <7dfebbd3-85d6-c7b7-b83b-fae8b644649e@yuripv.net> <478965aa-5256-e356-5339-de6fb82c3459@selasky.org> <63daa36a-5c22-6b08-3cd7-562fa961ab61@yuripv.net> <7f6de96d-8b56-e242-8950-04a20b197bce@selasky.org> From: Yuri Pankov Message-ID: <311a21e3-ed61-8679-b416-b2a4c255c6e7@yuripv.net> Date: Wed, 28 Aug 2019 14:41:07 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <7f6de96d-8b56-e242-8950-04a20b197bce@selasky.org> Content-Type: multipart/mixed; boundary="------------DC60AD2380EEEBE0278B6A9E" Content-Language: en-US X-Rspamd-Queue-Id: 46JP2b2BF7z45kl X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yuripv.net header.s=fm1 header.b=At1JIRRT; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=OYkk18KU; dmarc=none; spf=pass (mx1.freebsd.org: domain of yuripv@yuripv.net designates 66.111.4.224 as permitted sender) smtp.mailfrom=yuripv@yuripv.net X-Spamd-Result: default: False [-5.17 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[yuripv.net:s=fm1,messagingengine.com:s=fm3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.224]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; DMARC_NA(0.00)[yuripv.net]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[yuripv.net:+,messagingengine.com:+]; CTYPE_MIXED_BOGUS(1.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.99)[-0.993,0]; MIME_BASE64_TEXT(0.10)[]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[224.4.111.66.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(-2.68)[ip: (-5.81), ipnet: 66.111.4.0/24(-4.84), asn: 11403(-2.68), country: US(-0.05)] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Aug 2019 11:41:12 -0000 This is a multi-part message in MIME format. --------------DC60AD2380EEEBE0278B6A9E Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Hans Petter Selasky wrote: > On 2019-08-28 11:44, Yuri Pankov wrote: >> Hans Petter Selasky wrote: >>> On 2019-08-28 11:07, Yuri Pankov wrote: >>>> I have a "timed sleep before timers are working" panic in ichsmb_readb() >>>> calling ichsmb_wait() which uses msleep(). That is trying to >>>> jedec_dimm(4) module so it's trying to attach pretty early in boot. >>>> What would be the correct replacement for msleep() here? >>>> >>> >>> If you only need a sleep-delay, pause() is the right one. It handles >>> cold-boot. >> >> I guess that won't work here as we need to be waked up by interrupt >> handler on command completion, and pause() seems to sleep >> unconditionally for the given time in 'cold' case (if I'm reading the >> code correctly). > > If you are too early inside a SYSINIT() path, then you cannot use > sleeping. You will have to use polling in a loop with a fixed DELAY() to > know the timeout. Thanks for the help. Something like the following (it seems to work)? --------------DC60AD2380EEEBE0278B6A9E Content-Type: text/plain; charset=UTF-8; name="ichsmb.diff.txt" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="ichsmb.diff.txt" ZGlmZiAtLWdpdCBhL3N5cy9kZXYvaWNoc21iL2ljaHNtYi5jIGIvc3lzL2Rldi9pY2hzbWIv aWNoc21iLmMKaW5kZXggNWZmNTRkYjg2MDAuLjUwMGRmMTg5MTU5IDEwMDY0NAotLS0gYS9z eXMvZGV2L2ljaHNtYi9pY2hzbWIuYworKysgYi9zeXMvZGV2L2ljaHNtYi9pY2hzbWIuYwpA QCAtNjA4LDcgKzYwOCw4IEBAIGljaHNtYl9kZXZpY2VfaW50cih2b2lkICpjb29raWUpCiAJ CQlzYy0+aWNoX2NtZCA9IC0xOwogCQkJYnVzX3dyaXRlXzEoc2MtPmlvX3JlcywKIAkJCSAg ICBJQ0hfSFNUX1NUQSwgc3RhdHVzKTsKLQkJCXdha2V1cChzYyk7CisJCQlpZiAoIWNvbGQp CisJCQkJd2FrZXVwKHNjKTsKIAkJCWJyZWFrOwogCQl9CiAKQEAgLTYzNyw4ICs2MzgsMjUg QEAgaWNoc21iX3dhaXQoc2NfcCBzYykKIAlLQVNTRVJUKHNjLT5pY2hfY21kICE9IC0xLAog CSAgICAoIiVzOiBpY2hfY21kPSVkXG4iLCBfX2Z1bmNfXyAsIHNjLT5pY2hfY21kKSk7CiAJ bXR4X2Fzc2VydCgmc2MtPm11dGV4LCBNQV9PV05FRCk7Ci0JZXJyb3IgPSBtc2xlZXAoc2Ms ICZzYy0+bXV0ZXgsIFBaRVJPLCAiaWNoc21iIiwgaHogLyA0KTsKLQlEQkcoIm1zbGVlcCAt PiAlZFxuIiwgZXJyb3IpOworCWlmIChjb2xkKSB7CisJCWNvbnN0IHVfaW50IHRpbWVvdXRf dXMgPSAyNTAwMDA7CisJCWNvbnN0IHVfaW50IGRlbGF5X3VzID0gNTAwOworCQlpbnQgaTsK KworCQllcnJvciA9IEVXT1VMREJMT0NLOworCQlmb3IgKGkgPSAwOyBpIDwgdGltZW91dF91 czsgaSArPSBkZWxheV91cykgeworCQkJbXR4X3VubG9jaygmc2MtPm11dGV4KTsKKwkJCURF TEFZKGRlbGF5X3VzKTsKKwkJCW10eF9sb2NrKCZzYy0+bXV0ZXgpOworCQkJaWYgKHNjLT5p Y2hfY21kID09IC0xKSB7CisJCQkJZXJyb3IgPSAwOworCQkJCWJyZWFrOworCQkJfQorCQl9 CisJfSBlbHNlIHsKKwkJZXJyb3IgPSBtc2xlZXAoc2MsICZzYy0+bXV0ZXgsIFBaRVJPLCAi aWNoc21iIiwgaHogLyA0KTsKKwkJREJHKCJtc2xlZXAgLT4gJWRcbiIsIGVycm9yKTsKKwl9 CiAJc3dpdGNoIChlcnJvcikgewogCWNhc2UgMDoKIAkJc21iX2Vycm9yID0gc2MtPnNtYl9l cnJvcjsK --------------DC60AD2380EEEBE0278B6A9E-- From owner-freebsd-hackers@freebsd.org Wed Aug 28 13:01:06 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2540CDC0CB for ; Wed, 28 Aug 2019 13:01:06 +0000 (UTC) (envelope-from yuripv@yuripv.net) Received: from new2-smtp.messagingengine.com (new2-smtp.messagingengine.com [66.111.4.224]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46JQpn10dBz49jx for ; Wed, 28 Aug 2019 13:01:04 +0000 (UTC) (envelope-from yuripv@yuripv.net) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailnew.nyi.internal (Postfix) with ESMTP id A9DC838AF; Wed, 28 Aug 2019 09:01:03 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Wed, 28 Aug 2019 09:01:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yuripv.net; h= subject:from:to:references:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=fm1; bh=I q5ivkdsVCeNNhAPzsfhY8qab9j1Coc39LUM4gmK9lU=; b=N2E/Emr6Yq0SnCrT/ RAhF1H5JDjqiz28NjMHzULnj3Pgu4BIXdPz11v4BsWqv4F8+pEhe7ck+9BmRTcvB J54Ra4c7PXf5ROnNzVFlcqrPpaMeWg13flFKfYLgVl8hLIZUwx+HppYkvs9q/Ive 0Jpq6Lmj7XEQNNnL3mN/BDsFc/LvFLTvZVijZGZKkYP2rcDnGfdc7KuJ2nYJEjyz i8ptnf2zrIWJ/qORyTOTGNjoDE5jvASmXwhVwoi7miOjRbsLJrIUjloy32LC7R2v 0gEj+AFoM+PEr5qTb9bGBOL9ny04eLm1OmcuhzfhuYBitIH4AWkCye0TiKWT0YiG 4mDvw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=Iq5ivkdsVCeNNhAPzsfhY8qab9j1Coc39LUM4gmK9 lU=; b=sofbeQhR4O8f3554oKuYZptM3XnstjuZYrZWPREiKrCcfoa5ympZdqben mOR7nM/xh5nPFvbwgKo/8TYb+2hrm83jvBWh5IJ+7WOi5NpDLZ45m+feY1geUSgA LLfwX+AOJ+I8AYYOVlkvj7XoZYwUHakvHMC+ZM0A4OZej8PclMBsbo7e4UzZXMHX 1C+R4P/3cR26dBindtagrD4KN+v0XM8zIT/5LBGiyXGvhX3iz9Sz0qW3KHRqFHF0 vTTi0fSAU2QEtj7re57INPSDbAzLn+ZSih1mejgU07XSBSTMusy8KJh5YpGZ0wW2 P7qaq0J9AlBRMZS/Zb3SKxGYdPwhQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrudeitddgheekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefuhffvfhfkffgfgggjtgfgsehtje ertddtfeejnecuhfhrohhmpegjuhhrihcurfgrnhhkohhvuceohihurhhiphhvseihuhhr ihhpvhdrnhgvtheqnecuffhomhgrihhnpehfrhgvvggsshgurdhorhhgnecukfhppedule ehrddvtdeirddukeefrdduieejnecurfgrrhgrmhepmhgrihhlfhhrohhmpeihuhhrihhp vheshihurhhiphhvrdhnvghtnecuvehluhhsthgvrhfuihiivgeptd X-ME-Proxy: Received: from [192.168.1.2] (unknown [195.206.183.167]) by mail.messagingengine.com (Postfix) with ESMTPA id 428F080059; Wed, 28 Aug 2019 09:01:01 -0400 (EDT) Subject: Re: ichsmb(4) and msleep() From: Yuri Pankov To: Hans Petter Selasky , freebsd-hackers@freebsd.org References: <7dfebbd3-85d6-c7b7-b83b-fae8b644649e@yuripv.net> <478965aa-5256-e356-5339-de6fb82c3459@selasky.org> <63daa36a-5c22-6b08-3cd7-562fa961ab61@yuripv.net> <7f6de96d-8b56-e242-8950-04a20b197bce@selasky.org> <311a21e3-ed61-8679-b416-b2a4c255c6e7@yuripv.net> Message-ID: <1f91f7a6-050d-d690-d374-6b06950d2ce2@yuripv.net> Date: Wed, 28 Aug 2019 16:00:59 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <311a21e3-ed61-8679-b416-b2a4c255c6e7@yuripv.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46JQpn10dBz49jx X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yuripv.net header.s=fm1 header.b=N2E/Emr6; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=sofbeQhR; dmarc=none; spf=pass (mx1.freebsd.org: domain of yuripv@yuripv.net designates 66.111.4.224 as permitted sender) smtp.mailfrom=yuripv@yuripv.net X-Spamd-Result: default: False [-6.40 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[yuripv.net:s=fm1,messagingengine.com:s=fm3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.224]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[yuripv.net]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[yuripv.net:+,messagingengine.com:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.99)[-0.991,0]; IP_SCORE(-2.81)[ip: (-6.49), ipnet: 66.111.4.0/24(-4.84), asn: 11403(-2.68), country: US(-0.05)]; RCVD_IN_DNSWL_LOW(-0.10)[224.4.111.66.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Aug 2019 13:01:06 -0000 Yuri Pankov wrote: > Hans Petter Selasky wrote: >> On 2019-08-28 11:44, Yuri Pankov wrote: >>> Hans Petter Selasky wrote: >>>> On 2019-08-28 11:07, Yuri Pankov wrote: >>>>> I have a "timed sleep before timers are working" panic in ichsmb_readb() >>>>> calling ichsmb_wait() which uses msleep(). That is trying to >>>>> jedec_dimm(4) module so it's trying to attach pretty early in boot. >>>>> What would be the correct replacement for msleep() here? >>>>> >>>> >>>> If you only need a sleep-delay, pause() is the right one. It handles >>>> cold-boot. >>> >>> I guess that won't work here as we need to be waked up by interrupt >>> handler on command completion, and pause() seems to sleep >>> unconditionally for the given time in 'cold' case (if I'm reading the >>> code correctly). >> >> If you are too early inside a SYSINIT() path, then you cannot use >> sleeping. You will have to use polling in a loop with a fixed DELAY() to >> know the timeout. > > Thanks for the help. > > Something like the following (it seems to work)? Here's a review with the nit you mentioned fixed, thanks! https://reviews.freebsd.org/D21452 From owner-freebsd-hackers@freebsd.org Wed Aug 28 13:07:58 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 55E30DC3E8 for ; Wed, 28 Aug 2019 13:07:58 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46JQyj4Vyvz4BDq for ; Wed, 28 Aug 2019 13:07:57 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x72e.google.com with SMTP id 4so2287349qki.6 for ; Wed, 28 Aug 2019 06:07:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=DISCS1/CzBBSPv7EJVN4JB75IO1pSK26dN0u1jCtXPY=; b=L9iX3v5tTXgeHJaplrRorx2lNY4aejczOy1suf6AI2FcZAc2MJ9ljXYOaDjMZMvrIK 0tQxcByfwQUulpaups8dJqQErw7fz4KhgUcvkrsgGZikrnniwHXtlFI/ZOWg+xSh/tSc yuHah+hzaKLeZxkwhb1zgHkF/4NuQ3XutO1ZZW8qV8sF4bdYU788mOXnMbgmK0YSwd+6 Jbee1xlXwcMbxl2C2iC+JtZE8voN7mn5QQ5XMP3xYZKnyAFJ/s25Jj1Yh82v3+/cKxht HMiBVU41hOcvSApWJacgyNKxRoFCJSLSGFqtVn2K2xuvcuu+JRZwBX/+9tyQsAUB2G09 4rrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=DISCS1/CzBBSPv7EJVN4JB75IO1pSK26dN0u1jCtXPY=; b=dZUxlx8oipdoBdLqB0Ai2KkzI/E0ODlzHg+k3tSIA5EraGX7b/4osufXRDtZljipOR 2DHKvOlSjNQQ1b6djrh0n2S4Uw5B4ZVNt1g7JR7UI1zdJKVl+kjd1XIoXkfiptsDSm2V LJ1FzpAPA8TxOq2fBjCjp+I60xBw7uAOKXhexcV/mdo+kq/wgXKJBC4SfX+ogaynEPLE SGlePyTr4q/bkTcjbk5UdiIIVdE2NXCFbKWSOkCCjNY6di9XEdkJSa+KWtdBbdoxMtAp EWhEe2J1KWlF6224YOi04fBQjKRpRzRiDYFIEKu/l3gYwiiP8j6ckl0z4tHMJsDwLmsm pE4w== X-Gm-Message-State: APjAAAV00cPcE7qWYRqCnO/6aEzsa5IhZKOF1UUXkQcmJaiPx22A+5uP 0EBbaUK0cjTD7gQEQJ5fnmayjRSzSgoYsRLKaSP2kg== X-Google-Smtp-Source: APXvYqxRrcNHzwba9nGidZrqC0YWO8Mm7VRCkUFgx48MbsE5eiZM8cNpb9O9fymNyjFDMlERb/0dbfg8UHPVn16tfG8= X-Received: by 2002:a37:8902:: with SMTP id l2mr3654389qkd.380.1566997676307; Wed, 28 Aug 2019 06:07:56 -0700 (PDT) MIME-Version: 1.0 References: <7dfebbd3-85d6-c7b7-b83b-fae8b644649e@yuripv.net> <478965aa-5256-e356-5339-de6fb82c3459@selasky.org> <63daa36a-5c22-6b08-3cd7-562fa961ab61@yuripv.net> <7f6de96d-8b56-e242-8950-04a20b197bce@selasky.org> <311a21e3-ed61-8679-b416-b2a4c255c6e7@yuripv.net> <1f91f7a6-050d-d690-d374-6b06950d2ce2@yuripv.net> In-Reply-To: <1f91f7a6-050d-d690-d374-6b06950d2ce2@yuripv.net> From: Warner Losh Date: Wed, 28 Aug 2019 07:07:44 -0600 Message-ID: Subject: Re: ichsmb(4) and msleep() To: Yuri Pankov Cc: Hans Petter Selasky , FreeBSD Hackers X-Rspamd-Queue-Id: 46JQyj4Vyvz4BDq X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=L9iX3v5t; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::72e) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-5.91 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-0.996,0]; RCVD_IN_DNSWL_NONE(0.00)[e.2.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.92)[ip: (-9.35), ipnet: 2607:f8b0::/32(-2.85), asn: 15169(-2.32), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Aug 2019 13:07:58 -0000 On Wed, Aug 28, 2019, 7:01 AM Yuri Pankov wrote: > Yuri Pankov wrote: > > Hans Petter Selasky wrote: > >> On 2019-08-28 11:44, Yuri Pankov wrote: > >>> Hans Petter Selasky wrote: > >>>> On 2019-08-28 11:07, Yuri Pankov wrote: > >>>>> I have a "timed sleep before timers are working" panic in > ichsmb_readb() > >>>>> calling ichsmb_wait() which uses msleep(). That is trying to > >>>>> jedec_dimm(4) module so it's trying to attach pretty early in boot. > >>>>> What would be the correct replacement for msleep() here? > >>>>> > >>>> > >>>> If you only need a sleep-delay, pause() is the right one. It handles > >>>> cold-boot. > >>> > >>> I guess that won't work here as we need to be waked up by interrupt > >>> handler on command completion, and pause() seems to sleep > >>> unconditionally for the given time in 'cold' case (if I'm reading the > >>> code correctly). > >> > >> If you are too early inside a SYSINIT() path, then you cannot use > >> sleeping. You will have to use polling in a loop with a fixed DELAY() > to > >> know the timeout. > > > > Thanks for the help. > > > > Something like the following (it seems to work)? > > Here's a review with the nit you mentioned fixed, thanks! > > https://reviews.freebsd.org/D21452 What's the advantages of doing this instead of deferring attach until the interrupts are running? Warner > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@freebsd.org Wed Aug 28 13:34:52 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3387DDCE8E for ; Wed, 28 Aug 2019 13:34:52 +0000 (UTC) (envelope-from yuripv@yuripv.net) Received: from new2-smtp.messagingengine.com (new2-smtp.messagingengine.com [66.111.4.224]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46JRYl2p1Cz4Cgx for ; Wed, 28 Aug 2019 13:34:51 +0000 (UTC) (envelope-from yuripv@yuripv.net) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailnew.nyi.internal (Postfix) with ESMTP id 9A06C3986; Wed, 28 Aug 2019 09:34:50 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Wed, 28 Aug 2019 09:34:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yuripv.net; h= subject:to:cc:references:from:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=fm1; bh=w DqRu2m/Er3AisAK3GcZJ8nkk7uDZ//nq8+hii7JNQY=; b=fyy6wkVouAzIPxCia CQFHWh7akw0WSXB/ZYD7LMJ/DYaYdYjUxXxCetPAZ6iWZ9Xg3EmnjmrNWH/IKjuq f1cTC2xNRafz5er1CACOr+lBmFmFL/LFjDzHvSPCgpi0DAX+rfMTh4N9vLVlqn/I 950fCzMqxWCOPkR8FMzPHGK6HugzBMSRNIxcShaZqldwlzw8PQjR7LVDC3oupWRo fSREfqifm1VPYEmdPfHYICY42ZsYMrDWoNa7V/Yv01VcOkQCkLdqi8023KZVvIXX dx3B61ThjkPUxYMdoOzfaQeZDkl9WHJb8xf6fbyqXFPRhl05cZiaorKTX67OZxYi /iChw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=wDqRu2m/Er3AisAK3GcZJ8nkk7uDZ//nq8+hii7JN QY=; b=KisdpYit53bDc/79y4sqbcPIJC4t29uLRVovYeoNCDXdY2ej/Qe2PHVoh Rg38aFzSF6B4zR3WDGp85ATQA5Dt1qoHVMhy3JidiL7TJOm7xIzgDxzIczJCqEnM tYeGcK8RmBKqyJjVroW1JfpBBvAs65/XRXiE9xGYmgNyxM1e8eES3PU+/AbnaJEA mBjPLxXnSxeoQYx2KISVobKTQK7C5VXH9ZMT7EqwQ0T7HauE+2tfMge87MIl2XHt pTfibwH8SZDcWScteG6WWddcBSwRLxrYGpKE/HzXRg2fpPtJCIobrlsaJPMfjrxu TVy6ixe2jzk0wPceoOMq4aZmU7VbQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrudeitddgieeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepuffvfhfhkffffgggjggtgfesthejredttdefjeenucfhrhhomhepjghurhhi ucfrrghnkhhovhcuoeihuhhrihhpvheshihurhhiphhvrdhnvghtqeenucffohhmrghinh epfhhrvggvsghsugdrohhrghenucfkphepudelhedrvddtiedrudekfedrudeijeenucfr rghrrghmpehmrghilhhfrhhomhephihurhhiphhvseihuhhrihhpvhdrnhgvthenucevlh hushhtvghrufhiiigvpedt X-ME-Proxy: Received: from [192.168.1.2] (unknown [195.206.183.167]) by mail.messagingengine.com (Postfix) with ESMTPA id 6C87E80066; Wed, 28 Aug 2019 09:34:48 -0400 (EDT) Subject: Re: ichsmb(4) and msleep() To: Warner Losh Cc: Hans Petter Selasky , FreeBSD Hackers References: <7dfebbd3-85d6-c7b7-b83b-fae8b644649e@yuripv.net> <478965aa-5256-e356-5339-de6fb82c3459@selasky.org> <63daa36a-5c22-6b08-3cd7-562fa961ab61@yuripv.net> <7f6de96d-8b56-e242-8950-04a20b197bce@selasky.org> <311a21e3-ed61-8679-b416-b2a4c255c6e7@yuripv.net> <1f91f7a6-050d-d690-d374-6b06950d2ce2@yuripv.net> From: Yuri Pankov Message-ID: <4201916c-0f7f-0bf6-2d17-e9f6a1879ba7@yuripv.net> Date: Wed, 28 Aug 2019 16:34:47 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46JRYl2p1Cz4Cgx X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yuripv.net header.s=fm1 header.b=fyy6wkVo; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=KisdpYit; dmarc=none; spf=pass (mx1.freebsd.org: domain of yuripv@yuripv.net designates 66.111.4.224 as permitted sender) smtp.mailfrom=yuripv@yuripv.net X-Spamd-Result: default: False [-6.51 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[yuripv.net:s=fm1,messagingengine.com:s=fm3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.224:c]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[yuripv.net]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yuripv.net:+,messagingengine.com:+]; NEURAL_HAM_SHORT(-0.99)[-0.990,0]; IP_SCORE(-2.92)[ip: (-7.02), ipnet: 66.111.4.0/24(-4.84), asn: 11403(-2.68), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[224.4.111.66.list.dnswl.org : 127.0.5.1] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Aug 2019 13:34:52 -0000 Warner Losh wrote: > On Wed, Aug 28, 2019, 7:01 AM Yuri Pankov wrote: > >> Yuri Pankov wrote: >>> Hans Petter Selasky wrote: >>>> On 2019-08-28 11:44, Yuri Pankov wrote: >>>>> Hans Petter Selasky wrote: >>>>>> On 2019-08-28 11:07, Yuri Pankov wrote: >>>>>>> I have a "timed sleep before timers are working" panic in >> ichsmb_readb() >>>>>>> calling ichsmb_wait() which uses msleep(). That is trying to >>>>>>> jedec_dimm(4) module so it's trying to attach pretty early in boot. >>>>>>> What would be the correct replacement for msleep() here? >>>>>>> >>>>>> >>>>>> If you only need a sleep-delay, pause() is the right one. It handles >>>>>> cold-boot. >>>>> >>>>> I guess that won't work here as we need to be waked up by interrupt >>>>> handler on command completion, and pause() seems to sleep >>>>> unconditionally for the given time in 'cold' case (if I'm reading the >>>>> code correctly). >>>> >>>> If you are too early inside a SYSINIT() path, then you cannot use >>>> sleeping. You will have to use polling in a loop with a fixed DELAY() >> to >>>> know the timeout. >>> >>> Thanks for the help. >>> >>> Something like the following (it seems to work)? >> >> Here's a review with the nit you mentioned fixed, thanks! >> >> https://reviews.freebsd.org/D21452 > > > > What's the advantages of doing this instead of deferring attach until the > interrupts are running? None that I can think of, just going with what was suggested and seeing other drivers doing the same. Could you please name a driver that defers attach until !cold? From owner-freebsd-hackers@freebsd.org Wed Aug 28 15:10:34 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F2AE2DF390 for ; Wed, 28 Aug 2019 15:10:34 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound2m.ore.mailhop.org (outbound2m.ore.mailhop.org [54.149.155.156]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46JThB3qSJz4KLF for ; Wed, 28 Aug 2019 15:10:34 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1567005032; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=Bz/5zZEBiOJmfLtTTu1GxTjwwSYmBRmTpGyIq3y94HpVgh42Q+wjjmD6tJ6OBOqmEZ3rgT7BARYBU BOaGop07Hhkx88aGY4DSP466OOwxn+7T9V6cbQohZRBfg4TC/ySVWfqwtRbeKvKJtFmGWP+N6IFgAs qbQlUGbQx3jqYgJMMLan7Vq2iLIaxNPgP7/vlTZHWEKuHSerCYc5IkcHyjusOQFZ0gC5DHDkdby2hx kqcuC8cAaOsHGdWnpzFQd4psm9zql9LplliYxv4wT51wk/1CjA2M/fRUexWj2DAqc6+vUuukyAC5Le KO8uBdNGrnsv5EENpTpC08ET2iet+lA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:to:from:subject:message-id:dkim-signature:from; bh=+ddXZEC8+IfwYKdSTfqKGVbi2NKyditxNmoxXdclZ/E=; b=wCRrSc2WQZQKoXj4Eg64Hr1X7DbYJm3e4bpvhNm/l3o5VAuljekqQI4fM6kFM+RIbTpVI9ZKDtNt6 PPIwoRm4SUOKn6GE0L7gtZpwz56n2ImZUpe7k+xzLoYbUu7u4TO7Aiqf4lDDUQcEb0E24NHijsR7A6 wSQNplanTzjr4GoQZRYORBJpJJ6EH9lQvXkHc6VfKta6TPnFzh687BPVEJZcsNmE0BOLZiw8U1wvV8 3GZGK5qwNJxxpW8DyyAVFMRN5Ka0jr1s9JUY9hVN3sbo0BMzbTBeml0UlOZMkqqtIFeAPS7cQJtmMO nkD0CzMg65J5C5Corl4lPuV+Of6pCRA== ARC-Authentication-Results: i=1; outbound4.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:to:from:subject:message-id:from; bh=+ddXZEC8+IfwYKdSTfqKGVbi2NKyditxNmoxXdclZ/E=; b=FHpUW3RpzBhAOjWa39GxO7MPAs95+fnJTaamXyVd6/Tbl/WKwh9mys1WoGj0mVnywR45Kp7j7eQx+ XNJ9xzOQqo+Bp1XF4/dvIyeMfIouheOLS5qJG6UGWvqqF6pUEjMauovhTfOehTPquORypSoyrO4dWE gNM9oO8iHQ/e3DEN3oq4lQPt5a6Ox1XPNj7CeG0TV9A2XvI8x/SVrbfxX+wJ4FPcI029dnr3pROxli fsG2tKDmDmIoJB6Hah+eKgQU9GULTCTjzVYcSpQ9G+dnJnxrY5l9eiq/tYZ3ui6NyHnTEa621q5y7r K53w0LT6mzEJgSlmctKacIIY5k+0laA== X-MHO-RoutePath: aGlwcGll X-MHO-User: f8653cea-c9a5-11e9-85ed-13b9aae3a1d2 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound4.ore.mailhop.org (Halon) with ESMTPSA id f8653cea-c9a5-11e9-85ed-13b9aae3a1d2; Wed, 28 Aug 2019 15:10:31 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x7SFASZ3013329; Wed, 28 Aug 2019 09:10:28 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <197d072dc73ec3b0426f3c4d4db8c1fdd9b124c8.camel@freebsd.org> Subject: Re: ichsmb(4) and msleep() From: Ian Lepore To: Yuri Pankov , Hans Petter Selasky , freebsd-hackers@freebsd.org Date: Wed, 28 Aug 2019 09:10:28 -0600 In-Reply-To: <311a21e3-ed61-8679-b416-b2a4c255c6e7@yuripv.net> References: <7dfebbd3-85d6-c7b7-b83b-fae8b644649e@yuripv.net> <478965aa-5256-e356-5339-de6fb82c3459@selasky.org> <63daa36a-5c22-6b08-3cd7-562fa961ab61@yuripv.net> <7f6de96d-8b56-e242-8950-04a20b197bce@selasky.org> <311a21e3-ed61-8679-b416-b2a4c255c6e7@yuripv.net> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46JThB3qSJz4KLF X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.983,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:16509, ipnet:54.148.0.0/15, country:US] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Aug 2019 15:10:35 -0000 On Wed, 2019-08-28 at 14:41 +0300, Yuri Pankov wrote: > Hans Petter Selasky wrote: > > On 2019-08-28 11:44, Yuri Pankov wrote: > > > Hans Petter Selasky wrote: > > > > On 2019-08-28 11:07, Yuri Pankov wrote: > > > > > I have a "timed sleep before timers are working" panic in > > > > > ichsmb_readb() > > > > > calling ichsmb_wait() which uses msleep(). That is trying to > > > > > jedec_dimm(4) module so it's trying to attach pretty early in > > > > > boot. > > > > > What would be the correct replacement for msleep() here? > > > > > > > > > > > > > If you only need a sleep-delay, pause() is the right one. It > > > > handles > > > > cold-boot. > > > > > > I guess that won't work here as we need to be waked up by > > > interrupt > > > handler on command completion, and pause() seems to sleep > > > unconditionally for the given time in 'cold' case (if I'm reading > > > the > > > code correctly). > > > > If you are too early inside a SYSINIT() path, then you cannot use > > sleeping. You will have to use polling in a loop with a fixed > > DELAY() to > > know the timeout. > > Thanks for the help. > > Something like the following (it seems to work)? > Do you actually need to communicate with these i2c slave devices as part of making the system usable early in boot? That sometimes happens on embedded systems where you need to communicate with a power- management system via i2c to bring up other devices. In that case you need to do polling until interrupts are available. If not, then this is probably fallout from a historic freebsd glitch where almost all i2c controller drivers attach the iicbus child before interrupts are available, and then rely on all the slave device drivers to not do any IO until interrupts are available. IMO, that's insane... a driver shouldn't attach children until it's in a state where it's able to service the IO requests from them. The simple fix is usually to change the last few lines of the controller's attach() code. Usually it's something like return bus_generic_attach(dev); and you can just change it to config_intrhook_oneshot((ich_func_t)bus_generic_attach, dev); return (0); -- Ian From owner-freebsd-hackers@freebsd.org Thu Aug 29 03:27:20 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0E0F0EEE88 for ; Thu, 29 Aug 2019 03:27:20 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-io1-f48.google.com (mail-io1-f48.google.com [209.85.166.48]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46Jp2H0wpbz46Hy for ; Thu, 29 Aug 2019 03:27:18 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-io1-f48.google.com with SMTP id d25so1441500iob.6 for ; Wed, 28 Aug 2019 20:27:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=n2mQQQPvZS+f5lPwa0zkwL9Dk0KGcUyBbWv5HmbRdY8=; b=Pi1CzqGy4Qb0FzY+zPaaf0a/nR8qSCrBrHY0l7kn5vMSA26Ct8Jrl5LnpTboUKqcc1 +vKAbA4a2f0bH3KO2U+wqiHn7bBpMXpPQ9r0a/X0+sXSB+TSy5+GnlRXpL7WEVCs+1jl 5AWLCO/cFEeVjF6PZpF85+oEfKcuZzrROolxeohCiEHEmZK8lnr4xKPmtS95e5ZPGfb0 iUSDM5bkldk5BZAmgJTvbEDaz9L5KCO9/DIRGLNYWBBwUcYSmlYB8zjhI+d7A3FlCn+e yVMhJOHvENxIAY8V65/ZvhxuBOar/ampKKtFf3LeBzd/cqde1Cop4MoQKnN+rIl5Ja1H Q4hw== X-Gm-Message-State: APjAAAViKrVjY7FRJ0rPz3wpsOGSMk7BiYrM7/R95QNaC/JI49S8OuCW 73IZio2T/atRzL7S27k4RazDGFls X-Google-Smtp-Source: APXvYqwKjTGZb1tQqe75ErDwQjizGn0Wt4+FCLpEWz3H+HlNHBKGozAO9Y3l7R4q+Uxe8zP0DYl9sA== X-Received: by 2002:a5d:8b8d:: with SMTP id p13mr8133995iol.62.1567049237520; Wed, 28 Aug 2019 20:27:17 -0700 (PDT) Received: from mail-io1-f41.google.com (mail-io1-f41.google.com. [209.85.166.41]) by smtp.gmail.com with ESMTPSA id w6sm766399iob.29.2019.08.28.20.27.16 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 28 Aug 2019 20:27:17 -0700 (PDT) Received: by mail-io1-f41.google.com with SMTP id z3so4062483iog.0 for ; Wed, 28 Aug 2019 20:27:16 -0700 (PDT) X-Received: by 2002:a02:b604:: with SMTP id h4mr7728121jam.78.1567049236478; Wed, 28 Aug 2019 20:27:16 -0700 (PDT) MIME-Version: 1.0 References: <7dfebbd3-85d6-c7b7-b83b-fae8b644649e@yuripv.net> <478965aa-5256-e356-5339-de6fb82c3459@selasky.org> <63daa36a-5c22-6b08-3cd7-562fa961ab61@yuripv.net> <7f6de96d-8b56-e242-8950-04a20b197bce@selasky.org> <311a21e3-ed61-8679-b416-b2a4c255c6e7@yuripv.net> <1f91f7a6-050d-d690-d374-6b06950d2ce2@yuripv.net> <4201916c-0f7f-0bf6-2d17-e9f6a1879ba7@yuripv.net> In-Reply-To: <4201916c-0f7f-0bf6-2d17-e9f6a1879ba7@yuripv.net> Reply-To: cem@freebsd.org From: Conrad Meyer Date: Wed, 28 Aug 2019 20:27:05 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: ichsmb(4) and msleep() To: Yuri Pankov Cc: Warner Losh , FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46Jp2H0wpbz46Hy X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of csecem@gmail.com designates 209.85.166.48 as permitted sender) smtp.mailfrom=csecem@gmail.com X-Spamd-Result: default: False [-5.62 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[cem@freebsd.org]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; REPLYTO_ADDR_EQ_FROM(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-0.996,0]; FORGED_SENDER(0.30)[cem@freebsd.org,csecem@gmail.com]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; TAGGED_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_NEQ_ENVFROM(0.00)[cem@freebsd.org,csecem@gmail.com]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[48.166.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.63)[ip: (-7.42), ipnet: 209.85.128.0/17(-3.34), asn: 15169(-2.32), country: US(-0.05)]; RWL_MAILSPIKE_POSSIBLE(0.00)[48.166.85.209.rep.mailspike.net : 127.0.0.17]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 03:27:20 -0000 On Wed, Aug 28, 2019 at 6:35 AM Yuri Pankov wrote: > > Warner Losh wrote: > > What's the advantages of doing this instead of deferring attach until the > > interrupts are running? > > None that I can think of, just going with what was suggested and seeing > other drivers doing the same. Could you please name a driver that > defers attach until !cold? I think pretty much all drivers attach when interrupts are enabled (not the same as !cold)? At least x86 enables interrupts on BSP at SI_SUB_INTR, and DRIVER_MODULE drivers *load* at SI_SUB_DRIVERS, and the INTR one is ordered before the other. My skim read is that drivers do not actually attach until SI_SUB_CONFIGURE. I think the panic / test in sleepq_set_timeout_sbt is maybe overly strong? !cold indicates the entire autoconfigure process has concluded. But interrupts are available long before that. Seems like hardclock is started at ~SI_SUB_CLOCKS? Which is admittedly after DRIVERS, but still long before !cold. I'm not sure what set of interrupt/timer functionality is needed for sleepq, but likely that condition can be relaxed. If it cannot be relaxed enough for your driver, you could expand your DRIVER_MODULE() into the expanded macro, replacing SI_SUB_DRIVERS with a later stage. Best, Conrad From owner-freebsd-hackers@freebsd.org Thu Aug 29 03:28:29 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 29BF8EF09E for ; Thu, 29 Aug 2019 03:28:29 +0000 (UTC) (envelope-from yuripv@yuripv.net) Received: from new3-smtp.messagingengine.com (new3-smtp.messagingengine.com [66.111.4.229]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46Jp3d0LkDz46Sd; Thu, 29 Aug 2019 03:28:28 +0000 (UTC) (envelope-from yuripv@yuripv.net) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailnew.nyi.internal (Postfix) with ESMTP id 5F4CE3ABE; Wed, 28 Aug 2019 23:28:28 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Wed, 28 Aug 2019 23:28:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yuripv.net; h= subject:to:cc:references:from:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=fm1; bh=/ 3FPaOvJv1vYX9XTW0LvY9XEt9NQy8JtKK3pBhHqUmU=; b=qZJMJ5Z2O4aNtabDq Ndzlke5gTJ+jtY+Z6eyYOEevoK/l7hiJPB6apbMDVBjtBjU/7f+FIeKSsrRZ62yP lrpJuqbErnmRe0xSXb+HtWFpRwn6Rsw5t5ZOLlYo1lMkCHF0rgeORIvwLJXChkwR Jdk4ukoYPVUbZySbZ9NEuClkwUGqei7i+qlZaMUO28mk+4F9Fl0EAO8iHLceEP/L tSLt4Doj3wGrLsPgjLWP3pVpbD0tPn1dQtT4xsrU0g3N+ffzBbbFNgsCgRP3OVHE H7HcZ0lfeRz6if9tJjaOaTsd59V0Mlr7CehrnLMSpP5FZMlbQzSwTq4ta+Y8/qiy OiawA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=/3FPaOvJv1vYX9XTW0LvY9XEt9NQy8JtKK3pBhHqU mU=; b=BkWPwh/1nQUCA2xez7KVZFcscWEYJBFWWSvp7CMKZ1POpmLoZFrjF4Up1 aw27smetXeN0HMDSVAr3KqPrg48ldlBNlHnfwre2Q1uocfML6lBj4OuoQSzgaGB3 xTnMU9h26vsmCSpy981RSNV8y4jId3HeXed4yEAgs7FWNQECKNfyqt7gt8WojCIO t7bq2hkpJIRtg17B+4Crsuj+C1pwBKfpObJQHe4umNvLHOupeJNY4X8H5ARP+6Aw Z0aL2sF5qb6deK91n8IKph4RizAXxQZ/Wmf8kFj4XGBQysbmMn1qpUlwu9sTW/7j AKJxLJqjQjVd7vxS7DMW7vyN+/9tg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrudeiuddgjedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepuffvfhfhkffffgggjggtgfesthejredttdefjeenucfhrhhomhepjghurhhi ucfrrghnkhhovhcuoeihuhhrihhpvheshihurhhiphhvrdhnvghtqeenucffohhmrghinh epfhhrvggvsghsugdrohhrghenucfkphepudelhedrvddtiedrudekfedrudeijeenucfr rghrrghmpehmrghilhhfrhhomhephihurhhiphhvseihuhhrihhpvhdrnhgvthenucevlh hushhtvghrufhiiigvpedt X-ME-Proxy: Received: from [192.168.1.2] (unknown [195.206.183.167]) by mail.messagingengine.com (Postfix) with ESMTPA id 51E8F8005A; Wed, 28 Aug 2019 23:28:27 -0400 (EDT) Subject: Re: ichsmb(4) and msleep() To: cem@freebsd.org Cc: Warner Losh , FreeBSD Hackers References: <7dfebbd3-85d6-c7b7-b83b-fae8b644649e@yuripv.net> <478965aa-5256-e356-5339-de6fb82c3459@selasky.org> <63daa36a-5c22-6b08-3cd7-562fa961ab61@yuripv.net> <7f6de96d-8b56-e242-8950-04a20b197bce@selasky.org> <311a21e3-ed61-8679-b416-b2a4c255c6e7@yuripv.net> <1f91f7a6-050d-d690-d374-6b06950d2ce2@yuripv.net> <4201916c-0f7f-0bf6-2d17-e9f6a1879ba7@yuripv.net> From: Yuri Pankov Message-ID: <950652dd-1020-a27c-bb2b-fb3bbeea50bf@yuripv.net> Date: Thu, 29 Aug 2019 06:28:25 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46Jp3d0LkDz46Sd X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.96 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.96)[-0.964,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 03:28:29 -0000 Conrad Meyer wrote: > On Wed, Aug 28, 2019 at 6:35 AM Yuri Pankov wrote: >> >> Warner Losh wrote: >>> What's the advantages of doing this instead of deferring attach until the >>> interrupts are running? >> >> None that I can think of, just going with what was suggested and seeing >> other drivers doing the same. Could you please name a driver that >> defers attach until !cold? > > I think pretty much all drivers attach when interrupts are enabled > (not the same as !cold)? At least x86 enables interrupts on BSP at > SI_SUB_INTR, and DRIVER_MODULE drivers *load* at SI_SUB_DRIVERS, and > the INTR one is ordered before the other. My skim read is that > drivers do not actually attach until SI_SUB_CONFIGURE. > > I think the panic / test in sleepq_set_timeout_sbt is maybe overly > strong? !cold indicates the entire autoconfigure process has > concluded. But interrupts are available long before that. Seems like > hardclock is started at ~SI_SUB_CLOCKS? Which is admittedly after > DRIVERS, but still long before !cold. I'm not sure what set of > interrupt/timer functionality is needed for sleepq, but likely that > condition can be relaxed. > > If it cannot be relaxed enough for your driver, you could expand your > DRIVER_MODULE() into the expanded macro, replacing SI_SUB_DRIVERS with > a later stage. Yes, thank you. I guess it's already sorted out with Ian's help, please see the review: https://reviews.freebsd.org/D21452 From owner-freebsd-hackers@freebsd.org Thu Aug 29 11:41:07 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8329DD302F; Thu, 29 Aug 2019 11:41:07 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46K1022Lh0z4XmM; Thu, 29 Aug 2019 11:41:05 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x7TBevg5062817 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 29 Aug 2019 14:41:00 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x7TBevg5062817 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x7TBevqY062816; Thu, 29 Aug 2019 14:40:57 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 29 Aug 2019 14:40:57 +0300 From: Konstantin Belousov To: Li-Wen Hsu Cc: fcp@freebsd.org, FreeBSD Hackers Subject: Re: FCP 20190401-ci_policy: CI policy Message-ID: <20190829114057.GZ71821@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-Rspamd-Queue-Id: 46K1022Lh0z4XmM X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-2.93 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; R_SPF_SOFTFAIL(0.00)[~all:c]; IP_SCORE_FREEMAIL(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.93)[-0.928,0]; IP_SCORE(0.00)[ip: (-2.60), ipnet: 2001:470::/32(-4.45), asn: 6939(-3.10), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; FREEMAIL_ENVFROM(0.00)[gmail.com]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 11:41:07 -0000 On Wed, Aug 28, 2019 at 12:29:58PM +0800, Li-Wen Hsu wrote: > It seems I was doing wrong that just changed the content of this FCP > to "feedback", but did not send to the right mailing lists. > > So I would like to make an announcement that the FCP > 20190401-ci_policy "CI policy": > > https://github.com/freebsd/fcp/blob/master/fcp-20190401-ci_policy.md > > is officially in "feedback" state to hopefully receive more comments > and suggestions, then we can move on for the next FCP state. What problem does the document tries to solve ? Or rather, do we really have the problem that it claims to solve ? >From my experience, normal peer pressure is enough to get things fixed quickly when it is possible to fix them quickly. If there is something more non-trivial, esp. in the tests and not the build, I am sure that a rule allowing anybody to do blind revert is much more harmful than having a test broken. More, I know that tests are of very low quality, which means that brokeness of the tests is not an indicator of anything until root cause is identified. Can we rely on the common sense of developers until there is indeed the visible problem ? From owner-freebsd-hackers@freebsd.org Thu Aug 29 12:03:03 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4258ED4542; Thu, 29 Aug 2019 12:03:03 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46K1TM0Ssxz4ZQF; Thu, 29 Aug 2019 12:03:03 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.codepro.be", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id BD55A1A4ED; Thu, 29 Aug 2019 12:03:02 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [10.0.2.193] (ptr-8rh08k12jlyqauo4swr.18120a2.ip6.access.telenet.be [IPv6:2a02:1811:240e:402:c59e:f85c:1ed8:db5b]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id 59CDD3817F; Thu, 29 Aug 2019 14:03:01 +0200 (CEST) From: "Kristof Provost" To: "Konstantin Belousov" Cc: "Li-Wen Hsu" , "FreeBSD Hackers" , fcp@freebsd.org Subject: Re: FCP 20190401-ci_policy: CI policy Date: Thu, 29 Aug 2019 14:03:00 +0200 X-Mailer: MailMate (2.0BETAr6137) Message-ID: <412537DD-D98F-4B92-85F5-CB93CF33F281@FreeBSD.org> In-Reply-To: <20190829114057.GZ71821@kib.kiev.ua> References: <20190829114057.GZ71821@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 12:03:03 -0000 On 29 Aug 2019, at 13:40, Konstantin Belousov wrote: > On Wed, Aug 28, 2019 at 12:29:58PM +0800, Li-Wen Hsu wrote: >> It seems I was doing wrong that just changed the content of this FCP >> to "feedback", but did not send to the right mailing lists. >> >> So I would like to make an announcement that the FCP >> 20190401-ci_policy "CI policy": >> >> https://github.com/freebsd/fcp/blob/master/fcp-20190401-ci_policy.md >> >> is officially in "feedback" state to hopefully receive more comments >> and suggestions, then we can move on for the next FCP state. > > What problem does the document tries to solve ? Or rather, do we > really > have the problem that it claims to solve ? > There are, somewhat regularly, commits which break functionality, or at the very least tests. The main objective of this policy proposal is to try to improve overall code quality by encouraging and empowering all committers to investigate and fix test failures. >> From my experience, normal peer pressure is enough to get things >> fixed > quickly when it is possible to fix them quickly. If there is something > more non-trivial, esp. in the tests and not the build, I am sure that > a rule allowing anybody to do blind revert is much more harmful than > having a test broken. > > More, I know that tests are of very low quality, which means that > brokeness of the tests is not an indicator of anything until root > cause > is identified. > I’m not sure I agree with the characterisation that the tests are of low quality. My own experience with the pf tests is that they test a large section of the network stack and firewall code. They’ve identified several very really issues (both pre- and post commit on the epoch-isatin of the network stack, for example, as well as a fairly important issue with IPv6 reassembly). It’s certainly true that the pf tests often reveal issues that are not in pf but in other code. I wouldn’t agree that this is a sign of low quality tests, but instead I consider it a sign that we don’t have enough tests for the network stack itself. > Can we rely on the common sense of developers until there is indeed > the > visible problem ? > I don’t want to suggest that people simply don’t care about test failures, because that’s clearly not true. On the other hand, I do think we can do better. There are at least two open problem in the network stack that I currently can’t get anyone to look at, and where I personally do not have sufficient context (or time) to fix them myself. (#239380, #238870). This proposal isn’t a silver bullet, I don’t think there is such a thing, but I do believe that elevating the visibility and importance of test failures can help us improve overall quality. Best regards, Kristof From owner-freebsd-hackers@freebsd.org Thu Aug 29 14:28:59 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 24FA4D72D6 for ; Thu, 29 Aug 2019 14:28:59 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound2m.ore.mailhop.org (outbound2m.ore.mailhop.org [54.149.155.156]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46K4jk2kdSz3FFh for ; Thu, 29 Aug 2019 14:28:57 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1567088936; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=fGy0RQ0sszQAPF/fBmm+bK0M1FIhVvdk4bQ2EuadAnrcjgbMM2wqY7NyLLd8JwqWKed78Abhlf82P yKx1cczkW5LK2te3j52S1d7i4k11NbJFPFJqI6bD/gYLTFx/rINGszOE7zftCPlM5vHJca8x6ftHuo 1x4T4r7yS4lz/b5FN0TbzOiTse3DUvIe3xhuUacIcvlXdA0ASAeaFtS6jPrc6yKcmh0mcx7ENNfj48 0+Qz1jYAdWFwipRMkfc77rFBPLUNaqOuIgWXT9TvQXS6tXcJcZHArxejrcy1u8+Wd9TiPQHJN2+1CW 4DsJQ16nS80iJq2hTE0oBorIGtV0JLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=NhfUR3+uWOkNIDXozDawCm9N7M+9jCNMCkXCaafNDkM=; b=FYSG3gg6yRfuYiudVHzcBXt0wcoB7Iyex9gVSBiSZjox7pzQKE+b5NNyBIc5clcHgQREkJusMl9qC Y4hkr5vZA2zH1fYIhOkwTaDyNYCROp2i8bhkYbdBA/sx6ZFo2dPGtF1ePT7Bmncb73MZNbTO7mgxRY SFkUn/RdDS7clmg50XfcD1xiUfKItDZ+EWrzFpKBMhnpBirhLPOzVf3Oij0VLXx5ysXJ1zcCrE8IHu v891YwyKuTxP7kL9phYZlxGm/ia+4qAv7cIqOFLOyR41/bSRPVXSbCL4nxGFhHksHGNladc1AM0xdg t4ZocP4FRyG/cxx8AVGU3j3HC47RVIA== ARC-Authentication-Results: i=1; outbound4.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=NhfUR3+uWOkNIDXozDawCm9N7M+9jCNMCkXCaafNDkM=; b=D/H/xoOpauWee2r4LPj58oxtoPg7FF3Rc/qCHCftiQdAgXIHJTEkG2cGYLYe2jjGrZ0FAXxQYWEG/ /ZvZ0QTqRuPkDhPR7CKneUOSBGGTKcVWaf0pV/HvQ9XqLO3byQ5D/tlg8LJqGJIsaYdMWF9efeim19 i+nPDi1+SoZhrLtHIuWLghR2L2ZbNu/WpLnttE4pNDOVZ9EKqq7j3l0kab4WUhcX6B6IIcT1hVBuPC duf/E+e3qvHbXLbSG0j9WE8O5ITP6PGLYkFkv0stzJQBlbVW0wgP0lKM6lApXWMhfd5xitL3bZKh9y Fr4QLH+ODBKysg53uzTs7ZjpGcSUgzg== X-MHO-RoutePath: aGlwcGll X-MHO-User: 5391e919-ca69-11e9-85ed-13b9aae3a1d2 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound4.ore.mailhop.org (Halon) with ESMTPSA id 5391e919-ca69-11e9-85ed-13b9aae3a1d2; Thu, 29 Aug 2019 14:28:54 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x7TESrlT017362; Thu, 29 Aug 2019 08:28:53 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: Subject: Re: ichsmb(4) and msleep() From: Ian Lepore To: cem@freebsd.org, Yuri Pankov Cc: FreeBSD Hackers Date: Thu, 29 Aug 2019 08:28:53 -0600 In-Reply-To: References: <7dfebbd3-85d6-c7b7-b83b-fae8b644649e@yuripv.net> <478965aa-5256-e356-5339-de6fb82c3459@selasky.org> <63daa36a-5c22-6b08-3cd7-562fa961ab61@yuripv.net> <7f6de96d-8b56-e242-8950-04a20b197bce@selasky.org> <311a21e3-ed61-8679-b416-b2a4c255c6e7@yuripv.net> <1f91f7a6-050d-d690-d374-6b06950d2ce2@yuripv.net> <4201916c-0f7f-0bf6-2d17-e9f6a1879ba7@yuripv.net> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46K4jk2kdSz3FFh X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; ASN(0.00)[asn:16509, ipnet:54.148.0.0/15, country:US] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 14:28:59 -0000 On Wed, 2019-08-28 at 20:27 -0700, Conrad Meyer wrote: > On Wed, Aug 28, 2019 at 6:35 AM Yuri Pankov wrote: > > > > Warner Losh wrote: > > > What's the advantages of doing this instead of deferring attach until the > > > interrupts are running? > > > > None that I can think of, just going with what was suggested and seeing > > other drivers doing the same. Could you please name a driver that > > defers attach until !cold? > > I think pretty much all drivers attach when interrupts are enabled > (not the same as !cold)? At least x86 enables interrupts on BSP at > SI_SUB_INTR, and DRIVER_MODULE drivers *load* at SI_SUB_DRIVERS, and > the INTR one is ordered before the other. My skim read is that > drivers do not actually attach until SI_SUB_CONFIGURE. > > I think the panic / test in sleepq_set_timeout_sbt is maybe overly > strong? !cold indicates the entire autoconfigure process has > concluded. But interrupts are available long before that. Seems like > hardclock is started at ~SI_SUB_CLOCKS? Which is admittedly after > DRIVERS, but still long before !cold. I'm not sure what set of > interrupt/timer functionality is needed for sleepq, but likely that > condition can be relaxed. > > If it cannot be relaxed enough for your driver, you could expand your > DRIVER_MODULE() into the expanded macro, replacing SI_SUB_DRIVERS with > a later stage. > > Afaik, only x86 enables interrupts before SI_SUB_CONFIGURE. Maybe sparc64 does too. Other arches do it as the last thing in SI_SUB_CONFIGURE, usually in a function named configure_final() in /autoconf.c. -- Ian From owner-freebsd-hackers@freebsd.org Thu Aug 29 14:42:30 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4F49ED78E4 for ; Thu, 29 Aug 2019 14:42:30 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound2m.ore.mailhop.org (outbound2m.ore.mailhop.org [54.149.155.156]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46K51L0dTFz3GCK for ; Thu, 29 Aug 2019 14:42:29 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1567089748; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=a/iixmRyANIz2XYKkMo+f6254i6U+Dj868uVtSpXjInXC50OteB6mcgXHseF4aKxlx59DaSO/LxHI WFlOWcyQVh3gQwBmVxoRllA0F8GdpXXZp3fPgWR09cBZjwYLGlMbJ3GqUypmRrToZSp0oGZomy82LA sAemnjbRJfsWI+Lpl0ADj1omv9lR28DiX9dYCaF+zOG4I0vZ0jkHwkApYXjwx2PBvxPhG/Ym4VsP3x lD/tynr3MChPNKUnT5NClfH5fq8mMvYGvDLvjCeUVF8/RUWDFeD0p09AAj1nRVHTNKEeXfeTqgzxEe EKa0MyJeLFJo17p7K8N9l6C0bu4e+7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=q0A5Af+FjYXzUKhHiX5T2y2vMEbwVSRnnMr5Zyj6Azc=; b=hgA0dvajySNQtFfXVtA3Y5N0iCTIJg+xGYAUYP7lpYNaDDdc5wW+rngfoMGY22zq0ZgNS3AGRv7xW xJE2CgvvbH9jm8J5iMRPFGE6TuUybtdVY2s5JWFaNUz+HvNtRosdmMWqcvT8xGD/xKMT3Dv+RACH3+ F9hVo+xUMvBpTQQm7OXVQB3kUpeUkIO36vilvo9RZDuQnsGVM9b3SDgdoF/rKXsLe6MWD3bBg+KcxA 2xkouo6xC/FUrkWaKgrdM2747hIPN/nqnKpwxg0eYUJ4ZriUik+UyqlenjqyvMt3S43HXUa1iNbvHe wNd9u9er+dBiEWL7gmpTsKOx1vcwjqw== ARC-Authentication-Results: i=1; outbound4.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=q0A5Af+FjYXzUKhHiX5T2y2vMEbwVSRnnMr5Zyj6Azc=; b=aQKiD4SUotZnnBulgPsFnO8j2mZHNV946DsBaHL7Fbvhzw4aaGrwPlkECSX2kYKd/nhC2Vmb5wDBu u55DpxWn+kAamJV9FoH1Wnp3D/TTcQmtNM4yoJbqb171jcS7EDKobQQU/SPBrzQAO48/yaY3WowBZR SzD9B9VKG+knKCLkZ2nmMAG8OS3GLVe0Xpot4P0c3W7te8XsLaxkOpJsn0zf3ATFuweU51qPVUXyNP 9Rg0Epo/Ra4WTrvWMVHX6VLoy/vpu4KTcdvVlCW3aI9b88AroL/pzn2Lx/lBc4mmDpfuQVpNQaMY8P +txhFAbszz3GsQlfhhoQYo6f5LPQNQA== X-MHO-RoutePath: aGlwcGll X-MHO-User: 3795f44d-ca6b-11e9-85ed-13b9aae3a1d2 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound4.ore.mailhop.org (Halon) with ESMTPSA id 3795f44d-ca6b-11e9-85ed-13b9aae3a1d2; Thu, 29 Aug 2019 14:42:26 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x7TEgP25017390; Thu, 29 Aug 2019 08:42:25 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <28934eb780342605090bf365ac3a2e0d522256f5.camel@freebsd.org> Subject: Re: FCP 20190401-ci_policy: CI policy From: Ian Lepore To: Konstantin Belousov , Li-Wen Hsu Cc: FreeBSD Hackers , fcp@freebsd.org Date: Thu, 29 Aug 2019 08:42:25 -0600 In-Reply-To: <20190829114057.GZ71821@kib.kiev.ua> References: <20190829114057.GZ71821@kib.kiev.ua> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46K51L0dTFz3GCK X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; ASN(0.00)[asn:16509, ipnet:54.148.0.0/15, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 14:42:30 -0000 On Thu, 2019-08-29 at 14:40 +0300, Konstantin Belousov wrote: > On Wed, Aug 28, 2019 at 12:29:58PM +0800, Li-Wen Hsu wrote: > > It seems I was doing wrong that just changed the content of this FCP > > to "feedback", but did not send to the right mailing lists. > > > > So I would like to make an announcement that the FCP > > 20190401-ci_policy "CI policy": > > > > https://github.com/freebsd/fcp/blob/master/fcp-20190401-ci_policy.md > > > > is officially in "feedback" state to hopefully receive more comments > > and suggestions, then we can move on for the next FCP state. > > What problem does the document tries to solve ? Or rather, do we really > have the problem that it claims to solve ? > > From my experience, normal peer pressure is enough to get things fixed > quickly when it is possible to fix them quickly. If there is something > more non-trivial, esp. in the tests and not the build, I am sure that > a rule allowing anybody to do blind revert is much more harmful than > having a test broken. > > More, I know that tests are of very low quality, which means that > brokeness of the tests is not an indicator of anything until root cause > is identified. > > Can we rely on the common sense of developers until there is indeed the > visible problem ? > I totally agree. This is an overly-bureaucratic solution in search of a problem. If this needs to be addressed at all (and I'm not sure it does), then another sentence or two in bullet item 10 in section 18.1 [*] of the committer's guide should be enough. And even then it needn't be overly-formal and should just mention that if a commit does break the build the committer is expected to be responsive to that problem and the commit might get reverted if they're unresponsive. I don't think we need schedules. (And I don't think breaking a test counts as breaking the build.) [*] https://www.freebsd.org/doc/en_US.ISO8859-1/articles/committers-guide/rules.html -- Ian From owner-freebsd-hackers@freebsd.org Thu Aug 29 14:42:40 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 85F9CD7913; Thu, 29 Aug 2019 14:42:40 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46K51W4wk8z3GG9; Thu, 29 Aug 2019 14:42:38 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x7TEgSpf005877 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 29 Aug 2019 17:42:31 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x7TEgSpf005877 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x7TEgS0o005876; Thu, 29 Aug 2019 17:42:28 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 29 Aug 2019 17:42:28 +0300 From: Konstantin Belousov To: Kristof Provost Cc: Li-Wen Hsu , FreeBSD Hackers , fcp@freebsd.org Subject: Re: FCP 20190401-ci_policy: CI policy Message-ID: <20190829144228.GA71821@kib.kiev.ua> References: <20190829114057.GZ71821@kib.kiev.ua> <412537DD-D98F-4B92-85F5-CB93CF33F281@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <412537DD-D98F-4B92-85F5-CB93CF33F281@FreeBSD.org> User-Agent: Mutt/1.12.1 (2019-06-15) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-Rspamd-Queue-Id: 46K51W4wk8z3GG9 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.91 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.91)[-0.910,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 14:42:40 -0000 On Thu, Aug 29, 2019 at 02:03:00PM +0200, Kristof Provost wrote: > On 29 Aug 2019, at 13:40, Konstantin Belousov wrote: > > On Wed, Aug 28, 2019 at 12:29:58PM +0800, Li-Wen Hsu wrote: > >> It seems I was doing wrong that just changed the content of this FCP > >> to "feedback", but did not send to the right mailing lists. > >> > >> So I would like to make an announcement that the FCP > >> 20190401-ci_policy "CI policy": > >> > >> https://github.com/freebsd/fcp/blob/master/fcp-20190401-ci_policy.md > >> > >> is officially in "feedback" state to hopefully receive more comments > >> and suggestions, then we can move on for the next FCP state. > > > > What problem does the document tries to solve ? Or rather, do we > > really > > have the problem that it claims to solve ? > > > There are, somewhat regularly, commits which break functionality, or at > the very least tests. > The main objective of this policy proposal is to try to improve overall > code quality by encouraging and empowering all committers to investigate > and fix test failures. But this policy does not encourage, if anything. It gives a free ticket to revert, discouraging committers. > > >> From my experience, normal peer pressure is enough to get things > >> fixed > > quickly when it is possible to fix them quickly. If there is something > > more non-trivial, esp. in the tests and not the build, I am sure that > > a rule allowing anybody to do blind revert is much more harmful than > > having a test broken. > > > > More, I know that tests are of very low quality, which means that > > brokeness of the tests is not an indicator of anything until root > > cause > > is identified. > > > I’m not sure I agree with the characterisation that the tests are of > low quality. My own experience with the pf tests is that they test a > large section of the network stack and firewall code. They’ve > identified several very really issues (both pre- and post commit on the > epoch-isatin of the network stack, for example, as well as a fairly > important issue with IPv6 reassembly). This is my experience with the tests for kernel/libc/libthr, most of which comes from contrib/netbsd-tests, as I understand. > It’s certainly true that the pf tests often reveal issues that are not > in pf but in other code. I wouldn’t agree that this is a sign of low > quality tests, but instead I consider it a sign that we don’t have > enough tests for the network stack itself. > > > Can we rely on the common sense of developers until there is indeed > > the > > visible problem ? > > > I don’t want to suggest that people simply don’t care about test > failures, because that’s clearly not true. > > On the other hand, I do think we can do better. There are at least two > open problem in the network stack that I currently can’t get anyone to > look at, and where I personally do not have sufficient context (or time) > to fix them myself. (#239380, #238870). > > This proposal isn’t a silver bullet, I don’t think there is such a > thing, but I do believe that elevating the visibility and importance of > test failures can help us improve overall quality. This is not what was proposed. I fully agree that build failures should be fixed ASAP, and the each test results change (note the formulation) should be examined. But I do not think we have the problem right now of a scale which requires the free pass to revert with 4h timer. From owner-freebsd-hackers@freebsd.org Thu Aug 29 14:44:51 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 86BE9D7BC8; Thu, 29 Aug 2019 14:44:51 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-lf1-x143.google.com (mail-lf1-x143.google.com [IPv6:2a00:1450:4864:20::143]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46K54331jvz3GZZ; Thu, 29 Aug 2019 14:44:51 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: by mail-lf1-x143.google.com with SMTP id r5so2760170lfc.3; Thu, 29 Aug 2019 07:44:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:to:cc; bh=vi6RluLvB3RVtOqMSwTBcCPHr3wMB6PjNtXI4mFu7xg=; b=CrqzX0hxzJTSQsxZAHboaUU8u3dqR11K6LpJ9jwdq73F9oUhj4CyNeuTNuEnZPnqRK VNrNsNiO9Qm25nw62K00nG3f3R2+b+Aq32tAYNPiRKyQq4WtEIvB2yb509mNJp2gyY4L B+djE2yXtmqcHFZmghtEOGO7kkbIQmhLMoNhaPEXuhMIydqrXASMw5eB9W3jMqZdGIAi w/UrGMPZpteOqaSkfbmWSG+iHfgqaY9HycWcvgv1ryyyGhB1LPEb2j2+uycQD68pOvA0 hyZcZ5JAUaSK0q+YdTafFjgB41LPbHDR3J8jU369ygq1BVxmdKsk+rw1CymZLtKWykru KrYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=vi6RluLvB3RVtOqMSwTBcCPHr3wMB6PjNtXI4mFu7xg=; b=dAtz8ut1UbkjdaNDJROUY/QJezK9em42OwfEJSWYxW8k1iqYNU+shgfM9zkl3TWX9K n65hlFVIhevM0BvB6/b0D3BPFpBorIMSXp4hDg0/r8DhmmcxPewFj63xzO9+gaFq5rrN 60bcRhS2IETfrb9zTR9sHQprjjkQYYzxX688nE5Q3kWJB1CO4xeo/SGMCF7GH+OfNWwm NVUMq+PvywV1gHrbG29/N267IeA1k4xdHQrBAzK+YERaWjukgTDfuXL9Cx/eCBhjN5mB KVF9+mqUG8B4NRzimvqYzYnVcWIqkE5vJ2gRIjwHgz7d9m7ndvbtBORzb3cWoaW5nmB7 wBzQ== X-Gm-Message-State: APjAAAVHgt6kUJT5rxAtZ3mIe+GU145Df2W/VmLKyVynYR4fBGpEP4gn mJtFu/G0+gWVJk255DejNPVboxZ97yF8cugEkcdlFeJ7DtU= X-Google-Smtp-Source: APXvYqyms0ZjwLVbkRFOQGGIENLSitckxYGyqhD4idJWirjrbzZH5VtzzWPtEQ2ybke4wT7q0ovg6JMYc79rGdWApz0= X-Received: by 2002:a19:674d:: with SMTP id e13mr6463979lfj.176.1567089889352; Thu, 29 Aug 2019 07:44:49 -0700 (PDT) MIME-Version: 1.0 References: <20190829114057.GZ71821@kib.kiev.ua> <412537DD-D98F-4B92-85F5-CB93CF33F281@FreeBSD.org> In-Reply-To: <412537DD-D98F-4B92-85F5-CB93CF33F281@FreeBSD.org> Reply-To: araujo@freebsd.org From: Marcelo Araujo Date: Thu, 29 Aug 2019 22:44:38 +0800 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Kristof Provost Cc: Konstantin Belousov , FreeBSD Hackers , Li-Wen Hsu , fcp@freebsd.org X-Rspamd-Queue-Id: 46K54331jvz3GZZ X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.98)[-0.983,0] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 14:44:51 -0000 Em qui, 29 de ago de 2019 =C3=A0s 20:03, Kristof Provost escreveu: > On 29 Aug 2019, at 13:40, Konstantin Belousov wrote: > > On Wed, Aug 28, 2019 at 12:29:58PM +0800, Li-Wen Hsu wrote: > >> It seems I was doing wrong that just changed the content of this FCP > >> to "feedback", but did not send to the right mailing lists. > >> > >> So I would like to make an announcement that the FCP > >> 20190401-ci_policy "CI policy": > >> > >> https://github.com/freebsd/fcp/blob/master/fcp-20190401-ci_policy.md > >> > >> is officially in "feedback" state to hopefully receive more comments > >> and suggestions, then we can move on for the next FCP state. > > > > What problem does the document tries to solve ? Or rather, do we > > really > > have the problem that it claims to solve ? > > > There are, somewhat regularly, commits which break functionality, or at > the very least tests. > The main objective of this policy proposal is to try to improve overall > code quality by encouraging and empowering all committers to investigate > and fix test failures. > Sure, but it doesn't sounds like you are empowering people to works in their spare time for the project, quite the opposite. Could you show something more feasible than "somewhat" regularly breaks functionality? Most of the time personally I'm running HEAD, built daily and I can't see this. > > >> From my experience, normal peer pressure is enough to get things > >> fixed > > quickly when it is possible to fix them quickly. If there is something > > more non-trivial, esp. in the tests and not the build, I am sure that > > a rule allowing anybody to do blind revert is much more harmful than > > having a test broken. > > > > More, I know that tests are of very low quality, which means that > > brokeness of the tests is not an indicator of anything until root > > cause > > is identified. > > > I=E2=80=99m not sure I agree with the characterisation that the tests are= of > low quality. My own experience with the pf tests is that they test a > large section of the network stack and firewall code. They=E2=80=99ve > identified several very really issues (both pre- and post commit on the > epoch-isatin of the network stack, for example, as well as a fairly > important issue with IPv6 reassembly). > It=E2=80=99s certainly true that the pf tests often reveal issues that ar= e not > in pf but in other code. I wouldn=E2=80=99t agree that this is a sign of = low > quality tests, but instead I consider it a sign that we don=E2=80=99t hav= e > enough tests for the network stack itself. > All the tests and CI are pretty new, or at least it is something new for most of everybody and people are getting used to that. I stop here, I would elaborate more, but after Ian's email, I think I don't need anymore. > > > Can we rely on the common sense of developers until there is indeed > > the > > visible problem ? > > > I don=E2=80=99t want to suggest that people simply don=E2=80=99t care abo= ut test > failures, because that=E2=80=99s clearly not true. > > On the other hand, I do think we can do better. There are at least two > open problem in the network stack that I currently can=E2=80=99t get anyo= ne to > look at, and where I personally do not have sufficient context (or time) > to fix them myself. (#239380, #238870). > > This proposal isn=E2=80=99t a silver bullet, I don=E2=80=99t think there = is such a > thing, but I do believe that elevating the visibility and importance of > test failures can help us improve overall quality. > > Best regards, > Kristof > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > --=20 --=20 Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org \/ \ ^ Power To Server. .\. /_) From owner-freebsd-hackers@freebsd.org Thu Aug 29 14:45:46 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0E025D7CDD for ; Thu, 29 Aug 2019 14:45:46 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x82f.google.com (mail-qt1-x82f.google.com [IPv6:2607:f8b0:4864:20::82f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46K5551snNz3Gk5 for ; Thu, 29 Aug 2019 14:45:45 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x82f.google.com with SMTP id t12so3919553qtp.9 for ; Thu, 29 Aug 2019 07:45:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=3CP8ffJcILN/c9QU/s8UPxyGicn9o+e8qLVU2MG60js=; b=GwYyDB+d5tdVdyXIB1XOlAKgajpGPLaE+/Ec0LX6dyBZl7qAllQvc9BiCE4mYWP0y4 EniMiHBflmzu+CqwS6oMsy9mhPiwybzFfYPmLAWwBVYnIu9YNkg3oOY7wsuEDt5ApePj qgaaD6V3W4aKHJE+K33RGN6GryQ0VJkpJh7NthWjVv5+JYUM7y4A5zhZqnQAzgJ0hbFl fzFQI93LJ+BdjqX5ic046SIqNdIVWL83pxurmFb1OudvZjCVm4A287Qv/B5r/7w+u9Fy 5Ttd8t2EvpyTYa4jdz1mv9Vpv5pvFe+ER7IoNRikJMGhJkMFCahVnlHjGODRhxJinRKR C8dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=3CP8ffJcILN/c9QU/s8UPxyGicn9o+e8qLVU2MG60js=; b=BfGVpyXF2FgU2wV8wy/nOLxZKRvtD1hE3yYWhsnHh3cy55l4NoddOfo6SfETjmB943 Q8+LsXVepE5gTAmnNDsTi1WY2kjUxzV+d0Ar9NN+4L/vEVLrAkJxaJcB3LGUj+euxdoI NNX2l+08BdblzvQjJU4A6gmei7ETm8HGoV4OiX/6Lsl6LvyMCpilPcoInaFUnRNYHR+t tS79sU8w90s1Q+kMhsJ0Iw8IaYFv5/buBu1y7f3IVw4ScoPks2APzZPHC+0+/9OG42rr OjOWBvhOthbctMkUmACrPkfpgQUBiTCcsZ65xaK4vboWGAVhkv5qUGqBKP/DIo9mPjna AusA== X-Gm-Message-State: APjAAAUS27V0X5gpDWMEQno7ddqExXAi2Hc7XNyNPj/xTqGwx1yJFfyy 6gE6jSqbPp1N1z3Ouk9v1587mj22vJoGd1ak5fLtxQ== X-Google-Smtp-Source: APXvYqwcDWcP7/c52zvFbpt9HHldKN8CwQp4lrPjqBgyO/kaSdCvC+n/7X4X1GC0AwR4JmkQItINGlg5bsUglSCII/8= X-Received: by 2002:a0c:f6c6:: with SMTP id d6mr6754679qvo.102.1567089943897; Thu, 29 Aug 2019 07:45:43 -0700 (PDT) MIME-Version: 1.0 References: <7dfebbd3-85d6-c7b7-b83b-fae8b644649e@yuripv.net> <478965aa-5256-e356-5339-de6fb82c3459@selasky.org> <63daa36a-5c22-6b08-3cd7-562fa961ab61@yuripv.net> <7f6de96d-8b56-e242-8950-04a20b197bce@selasky.org> <311a21e3-ed61-8679-b416-b2a4c255c6e7@yuripv.net> <1f91f7a6-050d-d690-d374-6b06950d2ce2@yuripv.net> <4201916c-0f7f-0bf6-2d17-e9f6a1879ba7@yuripv.net> In-Reply-To: From: Warner Losh Date: Thu, 29 Aug 2019 08:45:32 -0600 Message-ID: Subject: Re: ichsmb(4) and msleep() To: Ian Lepore Cc: "Conrad E. Meyer" , Yuri Pankov , FreeBSD Hackers X-Rspamd-Queue-Id: 46K5551snNz3Gk5 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=GwYyDB+d; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::82f) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-5.91 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-0.996,0]; RCVD_IN_DNSWL_NONE(0.00)[f.2.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.92)[ip: (-9.37), ipnet: 2607:f8b0::/32(-2.85), asn: 15169(-2.32), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 14:45:46 -0000 On Thu, Aug 29, 2019 at 8:29 AM Ian Lepore wrote: > On Wed, 2019-08-28 at 20:27 -0700, Conrad Meyer wrote: > > On Wed, Aug 28, 2019 at 6:35 AM Yuri Pankov wrote: > > > > > > Warner Losh wrote: > > > > What's the advantages of doing this instead of deferring attach > until the > > > > interrupts are running? > > > > > > None that I can think of, just going with what was suggested and seeing > > > other drivers doing the same. Could you please name a driver that > > > defers attach until !cold? > > > > I think pretty much all drivers attach when interrupts are enabled > > (not the same as !cold)? At least x86 enables interrupts on BSP at > > SI_SUB_INTR, and DRIVER_MODULE drivers *load* at SI_SUB_DRIVERS, and > > the INTR one is ordered before the other. My skim read is that > > drivers do not actually attach until SI_SUB_CONFIGURE. > > > > I think the panic / test in sleepq_set_timeout_sbt is maybe overly > > strong? !cold indicates the entire autoconfigure process has > > concluded. But interrupts are available long before that. Seems like > > hardclock is started at ~SI_SUB_CLOCKS? Which is admittedly after > > DRIVERS, but still long before !cold. I'm not sure what set of > > interrupt/timer functionality is needed for sleepq, but likely that > > condition can be relaxed. > > > > If it cannot be relaxed enough for your driver, you could expand your > > DRIVER_MODULE() into the expanded macro, replacing SI_SUB_DRIVERS with > > a later stage. > > > > > > Afaik, only x86 enables interrupts before SI_SUB_CONFIGURE. Maybe > sparc64 does too. Other arches do it as the last thing in > SI_SUB_CONFIGURE, usually in a function named configure_final() in > /autoconf.c. > Yes. In fact, I think this idiom is so wide spread, we'd be better off doing 'return (bus_delayed_attach_children(dev));' at the end to signal this and gather together all current uses in the tree under that. Generally, if you need interrupts enabled and scheduling running to configure your children, you should do that anyway. It's a convenient late point, and it's better to do all the late things at the same time with the same idiom than to try to hyper-optimize when they get done. We've cleaned up a number of messes where driver writers invent their own thing due to ignorance of the proper thing (or because there wasn't a recognition that it was similar to what lots of other drivers did). Warner From owner-freebsd-hackers@freebsd.org Thu Aug 29 14:47:24 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A5632D7E27; Thu, 29 Aug 2019 14:47:24 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-lj1-x243.google.com (mail-lj1-x243.google.com [IPv6:2a00:1450:4864:20::243]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46K56z687Cz3Gsd; Thu, 29 Aug 2019 14:47:23 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: by mail-lj1-x243.google.com with SMTP id x18so3352621ljh.1; Thu, 29 Aug 2019 07:47:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:to:cc; bh=qT+mBqubGxPBox80K474SvmrCGrQtlul6W9MHeIFwC8=; b=CytIhV5xnNr/yCULsKxKxTymlq6rTy8wwtnbqBsDp4YyMzNu3UDSaER18xohAb60NM g4mbQgmaJH87byh/ZWOBT3EdBs1Wj6EhC6O7VR3YMn+F5pjiGEkwBHQYr9KaOMrISAQH XIoZc5ZLsluAvkxXTh5NkDXqZMspQ3rbzSVhYwP/Jk4s0Xp4H9UfiyXRTH/9FoC95iIu CIBNK7KNDFMPmXPSzBcOSuIs9SbtVu5c1qfkTDRENP74lwt0klU49sx2P5A2dSzoUGp9 ZObLsEdvcdeFDW3BMhUbkEv00r9HUsW8VvngMhhGVusy7grwYaHkff7rhUMFsoDpdYSS l5Cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=qT+mBqubGxPBox80K474SvmrCGrQtlul6W9MHeIFwC8=; b=m0gQsAyR7SHyOORTzqDrhb1j1PtyYqRoK92Tm/ttv75tBpd6/AifN9yy3HTmhRPcBq rIn2LQ33gPWz5ht8F+KdLygcMbfz3i7JGsQYGCzGtCct69jCPkQBKD/OoTVkI5d+uSGG E6TkH59dKenKOD108n/LI9bwkfOOaVY8Fd2tSO8QCthZprKoXzi+78oHcpiAVQ+GEznP YmgEYyWfpvezwi5RI6AfFoh1lkXZvLkciZqsdc6BK3/GGyw0n+Kr6/faIe518J+048Rz sbdjbwnADTN4VVROF3sjgqzqfl+96VOpztGFJ2EBmDW+snI9SMGwjUEhCM1w6Oky3GpW Ru4Q== X-Gm-Message-State: APjAAAV4MsAFIgpyP9HQmlsAZm/kYxxNw/kwLz1OoI03cpTt6y0G9ssS YU1nTuO9T3wD/B+6rC2Uh+/DPfkOx+c2wYdBHG3DTJgzBQE= X-Google-Smtp-Source: APXvYqzhE8GwoiaGoth1QcocgSL9rLaPDRyZVlcvTprxsdI5quQEiQcpJHSVFcvcQEjYCKb5HJiF78TsSZxJWBUq1+o= X-Received: by 2002:a2e:81ca:: with SMTP id s10mr5649243ljg.181.1567090041764; Thu, 29 Aug 2019 07:47:21 -0700 (PDT) MIME-Version: 1.0 References: <20190829114057.GZ71821@kib.kiev.ua> <412537DD-D98F-4B92-85F5-CB93CF33F281@FreeBSD.org> In-Reply-To: Reply-To: araujo@freebsd.org From: Marcelo Araujo Date: Thu, 29 Aug 2019 22:47:10 +0800 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Kristof Provost Cc: Konstantin Belousov , FreeBSD Hackers , Li-Wen Hsu , fcp@freebsd.org X-Rspamd-Queue-Id: 46K56z687Cz3Gsd X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=CytIhV5x; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of araujobsdport@gmail.com designates 2a00:1450:4864:20::243 as permitted sender) smtp.mailfrom=araujobsdport@gmail.com X-Spamd-Result: default: False [-3.97 / 15.00]; HAS_REPLYTO(0.00)[araujo@freebsd.org]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; RCPT_COUNT_FIVE(0.00)[5]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.97)[-0.968,0]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (3.15), ipnet: 2a00:1450::/32(-2.99), asn: 15169(-2.32), country: US(-0.05)]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[3.4.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; FREEMAIL_CC(0.00)[gmail.com]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 14:47:24 -0000 Em qui, 29 de ago de 2019 =C3=A0s 22:44, Marcelo Araujo escreveu: > > > Em qui, 29 de ago de 2019 =C3=A0s 20:03, Kristof Provost > escreveu: > >> On 29 Aug 2019, at 13:40, Konstantin Belousov wrote: >> > On Wed, Aug 28, 2019 at 12:29:58PM +0800, Li-Wen Hsu wrote: >> >> It seems I was doing wrong that just changed the content of this FCP >> >> to "feedback", but did not send to the right mailing lists. >> >> >> >> So I would like to make an announcement that the FCP >> >> 20190401-ci_policy "CI policy": >> >> >> >> https://github.com/freebsd/fcp/blob/master/fcp-20190401-ci_policy.md >> >> >> >> is officially in "feedback" state to hopefully receive more comments >> >> and suggestions, then we can move on for the next FCP state. >> > >> > What problem does the document tries to solve ? Or rather, do we >> > really >> > have the problem that it claims to solve ? >> > >> There are, somewhat regularly, commits which break functionality, or at >> the very least tests. >> The main objective of this policy proposal is to try to improve overall >> code quality by encouraging and empowering all committers to investigate >> and fix test failures. >> > > Sure, but it doesn't sounds like you are empowering people to works in > their spare time for the project, quite the opposite. > Could you show something more feasible than "somewhat" regularly breaks > functionality? Most of the time personally I'm running HEAD, built daily > and I can't see this. > > >> >> >> From my experience, normal peer pressure is enough to get things >> >> fixed >> > quickly when it is possible to fix them quickly. If there is something >> > more non-trivial, esp. in the tests and not the build, I am sure that >> > a rule allowing anybody to do blind revert is much more harmful than >> > having a test broken. >> > >> > More, I know that tests are of very low quality, which means that >> > brokeness of the tests is not an indicator of anything until root >> > cause >> > is identified. >> > >> I=E2=80=99m not sure I agree with the characterisation that the tests ar= e of >> low quality. My own experience with the pf tests is that they test a >> large section of the network stack and firewall code. They=E2=80=99ve >> identified several very really issues (both pre- and post commit on the >> epoch-isatin of the network stack, for example, as well as a fairly >> important issue with IPv6 reassembly). >> It=E2=80=99s certainly true that the pf tests often reveal issues that a= re not >> in pf but in other code. I wouldn=E2=80=99t agree that this is a sign of= low >> quality tests, but instead I consider it a sign that we don=E2=80=99t ha= ve >> enough tests for the network stack itself. >> > > All the tests and CI are pretty new, or at least it is something new for > most of everybody and people are getting used to that. > > I stop here, I would elaborate more, but after Ian's email, I think I > don't need anymore. > One last thing I'd like to say: We should stop to try to control FreeBSD contributions, it is not doing good for the project. Let it be, let it go, there is no owner, just go and fix things. > > >> >> > Can we rely on the common sense of developers until there is indeed >> > the >> > visible problem ? >> > >> I don=E2=80=99t want to suggest that people simply don=E2=80=99t care ab= out test >> failures, because that=E2=80=99s clearly not true. >> >> On the other hand, I do think we can do better. There are at least two >> open problem in the network stack that I currently can=E2=80=99t get any= one to >> look at, and where I personally do not have sufficient context (or time) >> to fix them myself. (#239380, #238870). >> >> This proposal isn=E2=80=99t a silver bullet, I don=E2=80=99t think there= is such a >> thing, but I do believe that elevating the visibility and importance of >> test failures can help us improve overall quality. >> >> Best regards, >> Kristof >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.or= g >> " >> > > > -- > > -- > Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.Fr= eeBSD.org \/ \ ^ > Power To Server. .\. /_) > > --=20 --=20 Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org \/ \ ^ Power To Server. .\. /_) From owner-freebsd-hackers@freebsd.org Thu Aug 29 14:54:29 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 90849D8174; Thu, 29 Aug 2019 14:54:29 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46K5H92v8Mz3HSC; Thu, 29 Aug 2019 14:54:29 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.codepro.be", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id 162311BEA9; Thu, 29 Aug 2019 14:54:29 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [10.0.2.193] (ptr-8rh08k12jlyqauo4swr.18120a2.ip6.access.telenet.be [IPv6:2a02:1811:240e:402:c59e:f85c:1ed8:db5b]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id 798F73858E; Thu, 29 Aug 2019 16:54:27 +0200 (CEST) From: "Kristof Provost" To: "Ian Lepore" Cc: "Konstantin Belousov" , "Li-Wen Hsu" , "FreeBSD Hackers" , fcp@freebsd.org Subject: Re: FCP 20190401-ci_policy: CI policy Date: Thu, 29 Aug 2019 16:54:26 +0200 X-Mailer: MailMate (2.0BETAr6137) Message-ID: In-Reply-To: <28934eb780342605090bf365ac3a2e0d522256f5.camel@freebsd.org> References: <20190829114057.GZ71821@kib.kiev.ua> <28934eb780342605090bf365ac3a2e0d522256f5.camel@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 14:54:29 -0000 On 29 Aug 2019, at 16:42, Ian Lepore wrote: > (And I don't think breaking a test counts as > breaking the build.) > I fundamentally disagree on this point. A test failure is, just like a compiler warning, a precious gift that should not be ignored. The more distance (both in terms of time, and in terms of the people involved) there is between a bug being introduced and it being detected the harder it is to fix it. Test accelerate detection of bugs. If we do not take test failures seriously (i.e. as an indication something is wrong and should be fixed) the tests will inevitable become useless in one of two ways: we’ll either disable failing tests (which is what we tend to do now) reducing test coverage or we’ll have a test suite with many failures in it, which makes it useless as well. (As with compiler warnings, the best way to keep them under control is to consider them to be fatal errors.) In either scenario we end up reducing test coverage, which means we’re going to push more bugs towards users. > I totally agree. This is an overly-bureaucratic solution in search of > a problem. > > If this needs to be addressed at all (and I'm not sure it does), then > another sentence or two in bullet item 10 in section 18.1 [*] of the > committer's guide should be enough. And even then it needn't be > overly-formal and should just mention that if a commit does break the > build the committer is expected to be responsive to that problem and > the commit might get reverted if they're unresponsive. I don't think > we need schedules. > I do feel that’s a better argument. We’ve always had a policy of reverting on request (AIUI), so this is more or less trying to be a strong restatement of that, more than a fundamental shift in policy. Best regards, Kristof From owner-freebsd-hackers@freebsd.org Thu Aug 29 15:01:31 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7BDA7D86A3; Thu, 29 Aug 2019 15:01:31 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-lj1-x242.google.com (mail-lj1-x242.google.com [IPv6:2a00:1450:4864:20::242]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46K5RH2jN9z3J1S; Thu, 29 Aug 2019 15:01:30 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: by mail-lj1-x242.google.com with SMTP id x3so3371009lji.5; Thu, 29 Aug 2019 08:01:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:to:cc; bh=3ZnXP+zKpmB0LU+a5ZZE56+58oJErxdNO5Cw30lJ9Nc=; b=iyca442rNhBaC5GvY3I1J6rgHOTFCFbvwuCShgpJxVmZb74FqKKdh5IsPeyVEt6dCR oifbq6ShTCSnFsiQ07i3/j/FcQIGs7kzPfYWi5OmbQj6c51POInk6/2x3PstpgX3BKIR uXTDUv8qPwg726H4f+ZAkA99OXf2//jVd/TN4zRNHHJigUoKbiY+fzd5GuO7aWg7lCCB Y+MXnREBJ2nFOo/h1p1jU2Lbg+olpTuXjzUtvGFmLLRuMy9gEcG3AG/EvnkRCHmLsZA2 +LVmcv9XLTJkiCYgO7qQbCJFw/ZBmD4NTY8WUxsVRPbmtPcV3IKW/rMrYNCHc1mGi3mh EapQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=3ZnXP+zKpmB0LU+a5ZZE56+58oJErxdNO5Cw30lJ9Nc=; b=hENPB6nko6iBiLvVo8/4XF3UQddmaLI6vzsy9wDfsTc65xx4SF5Pj4hjPOCJwM0PIm Xsm3AbxiuSkvUXOiyi0Pi5kmHwIHpRE3Ot//sWfc4TY87h4XX0mbF0mLwj7dMA4gJXpL 1PEC1E4x+vh7Qn+iiMp4nypDImsgO4SLndnP2gpKPMAHK1DoZkqisCly+WzdgT27M93C 3Gl5+xFh7pdINtw4TDI7XulxETwki2+2pkmXb13y2TgkXDRLqJfUYB3Ws2JvbUjAFqN3 D4dx4HMPsjRnBgAZy8IZI53xczNxSXPkZYUl9OMJ18Ph5gK/5mBlwaRaR6Qkc6Wsg8ML ig6Q== X-Gm-Message-State: APjAAAVmzLZcsSBnLBJkyV5M6AgDn7VpOmwpDe0iOV64TASvXXrob4Zd WKvze9TAG/K9uDA0Tiq7fhO9IzJIo3G5BSl7v06UnYdmnPY= X-Google-Smtp-Source: APXvYqzv8wycrFM8IQuvkvHTJcg4qdljRE7bJUUQ1m0UndetIGa91MEK55ybN55nm3tOrLGoUh5OdxFsxtcAOpMGeSw= X-Received: by 2002:a2e:81ca:: with SMTP id s10mr5692168ljg.181.1567090889056; Thu, 29 Aug 2019 08:01:29 -0700 (PDT) MIME-Version: 1.0 References: <20190829114057.GZ71821@kib.kiev.ua> <28934eb780342605090bf365ac3a2e0d522256f5.camel@freebsd.org> In-Reply-To: Reply-To: araujo@freebsd.org From: Marcelo Araujo Date: Thu, 29 Aug 2019 23:01:17 +0800 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Kristof Provost Cc: Ian Lepore , Konstantin Belousov , FreeBSD Hackers , Li-Wen Hsu , fcp@freebsd.org X-Rspamd-Queue-Id: 46K5RH2jN9z3J1S X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 15:01:31 -0000 Em qui, 29 de ago de 2019 =C3=A0s 22:54, Kristof Provost escreveu: > On 29 Aug 2019, at 16:42, Ian Lepore wrote: > > (And I don't think breaking a test counts as > > breaking the build.) > > > I fundamentally disagree on this point. A test failure is, just like a > compiler warning, a precious gift that should not be ignored. > The more distance (both in terms of time, and in terms of the people > involved) there is between a bug being introduced and it being detected > the harder it is to fix it. Test accelerate detection of bugs. If we do > not take test failures seriously (i.e. as an indication something is > wrong and should be fixed) the tests will inevitable become useless in > one of two ways: we=E2=80=99ll either disable failing tests (which is wha= t we > tend to do now) reducing test coverage or we=E2=80=99ll have a test suite= with > many failures in it, which makes it useless as well. (As with compiler > warnings, the best way to keep them under control is to consider them to > be fatal errors.) > Could you elaborate where is the "fundamentally" you disagree? Where is the fundament? You guys are introducing something new, yes everybody knows about test, it is year 2019, but nobody can come with new rules tha in hours we gonna revert if you "dare to don't fix it". Sorry, this is not how people test software and fix it. > > In either scenario we end up reducing test coverage, which means we=E2=80= =99re > going to push more bugs towards users. > > > I totally agree. This is an overly-bureaucratic solution in search of > > a problem. > > > > If this needs to be addressed at all (and I'm not sure it does), then > > another sentence or two in bullet item 10 in section 18.1 [*] of the > > committer's guide should be enough. And even then it needn't be > > overly-formal and should just mention that if a commit does break the > > build the committer is expected to be responsive to that problem and > > the commit might get reverted if they're unresponsive. I don't think > > we need schedules. > > > I do feel that=E2=80=99s a better argument. We=E2=80=99ve always had a po= licy of > reverting on request (AIUI), so this is more or less trying to be a > strong restatement of that, more than a fundamental shift in policy. > We don't have a policy to revert commit, actually revert commit is something bad, it is kind of punishment, I have been there, nobody wants to be there. Stop to push this non-sense argument. > > Best regards, > Kristof > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > --=20 --=20 Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org \/ \ ^ Power To Server. .\. /_) From owner-freebsd-hackers@freebsd.org Thu Aug 29 15:02:50 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 944C8D88C1; Thu, 29 Aug 2019 15:02:50 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46K5Sp3LJKz3JJC; Thu, 29 Aug 2019 15:02:50 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.codepro.be", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id 25CDC1BFE8; Thu, 29 Aug 2019 15:02:50 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [10.0.2.193] (ptr-8rh08k12jlyqauo4swr.18120a2.ip6.access.telenet.be [IPv6:2a02:1811:240e:402:c59e:f85c:1ed8:db5b]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id A83F6385CC; Thu, 29 Aug 2019 17:02:48 +0200 (CEST) From: "Kristof Provost" To: "Konstantin Belousov" Cc: "Li-Wen Hsu" , "FreeBSD Hackers" , fcp@freebsd.org Subject: Re: FCP 20190401-ci_policy: CI policy Date: Thu, 29 Aug 2019 17:02:47 +0200 X-Mailer: MailMate (2.0BETAr6137) Message-ID: In-Reply-To: <20190829144228.GA71821@kib.kiev.ua> References: <20190829114057.GZ71821@kib.kiev.ua> <412537DD-D98F-4B92-85F5-CB93CF33F281@FreeBSD.org> <20190829144228.GA71821@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed; markup=markdown Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 15:02:50 -0000 On 29 Aug 2019, at 16:42, Konstantin Belousov wrote: > On Thu, Aug 29, 2019 at 02:03:00PM +0200, Kristof Provost wrote: >> There are, somewhat regularly, commits which break functionality, or >> at >> the very least tests. >> The main objective of this policy proposal is to try to improve >> overall >> code quality by encouraging and empowering all committers to >> investigate >> and fix test failures. > But this policy does not encourage, if anything. > It gives a free ticket to revert, discouraging committers. > To provide a counterpoint here: my personal frustration right now is that I’ve spent a good bit of time adding tests for pf and fixing bugs for it, only to see the tests having to be disabled because of unrelated (to pf) changes in the network stack. Either through lack of visibility, or lack of time, or because people assume pf tests failures must by definition be the responsibility of the pf maintainer, these failures have not been investigated by anyone other than me, and I lack the time and subject matter expertise to fix them. I’m desperately afraid that if/when these bugs do get fixed we’re going to discover that other things have broken in the mean time, and the tests are still going to fail, for different reasons. These are bugs. They’re the best case scenario for bug reports even, because they come with a reproduction case built-in, and yet they’re still not getting fixed. This too is discouraging. I’m open to alternative proposals for how to address that problem, but I don’t think that “continue on as we always have” is the correct answer. Best regards, Kristof From owner-freebsd-hackers@freebsd.org Thu Aug 29 15:08:50 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6EDF7D8ABE for ; Thu, 29 Aug 2019 15:08:50 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x833.google.com (mail-qt1-x833.google.com [IPv6:2607:f8b0:4864:20::833]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46K5bj4qwcz3JZB for ; Thu, 29 Aug 2019 15:08:49 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x833.google.com with SMTP id y26so4047409qto.4 for ; Thu, 29 Aug 2019 08:08:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=2Wc2bEFcg8n1w6AcR/y2lTDsyKi+cVQ4C3w3oWA0M4U=; b=PgDmVkDkJHnFZ1HuuhjK5rT3egZ6RQDY8qFiw6tRNUmX5Y+Q/xd7JHcymuynH7dX0c EiDyXRsKF3tPgjYALwY1x8gTZK8Tur5O3BkrN7DYuQ9yDfgIlSIePKTD905Kty6en3NV n3kyPJ5dC703+lJudbChMUHR1+GpQ0kKaVLq9cDx97SewRIUs1Fw6rFwJpG487ffRXXR Iquw6yXFLQTcAcD14Tm6/UHeCwnq1FJuK7tTSLS/hoEwOauGXMHQ5t61wUs7z2rSJzW4 nHQAwSmdpwM3E90mENjXVAbSsba46+lLlWl/jqJPzDNeLqkZvut1wOi8W9lObR/JJGWP WUxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=2Wc2bEFcg8n1w6AcR/y2lTDsyKi+cVQ4C3w3oWA0M4U=; b=m9sIXUUDfExfLiDmuzq1zCsQZReWO4qNi8TwD/9yFfoZsM8IBYrwLY4c15Ft18Ddd4 KD3uDSk4TyDrKNrfRlVysrdZN2EC2Wm2b0OXX+3aSNRntzE4pR6r/DUW7Z9YYq0VyzRs OvpOu82X0FQJj0Em2pyqC3GgYWCdmy8bW83pGBlSFjyaalMoEMhanLY7n/5pbFCtVMhL syVtjoLGw7T9MBeNZwLjyWrl5XhaKws5+NZoPSPKQZPzBk27x0c0MwH+1+20y9AuJhFP +k1HB4hZPWBrgkya22PeeVmy7J6cxhVg3k+6XgGEiAxmSiORBXPDx/txKcH9KN2jrgFf lNcQ== X-Gm-Message-State: APjAAAV0Qfmo3nNtbeRwu7DW3lD8LWskbW4ONat1z0XzjHWiM+ki7Nyv xgTT0mxKhuuT2yiE/4sJSPQYUtOM2T88rbIKOLVPvQ== X-Google-Smtp-Source: APXvYqw6prGe0bR/jCOJ4nFhjgXq7WN7WbMmL5Wx3W/Pfn7FVwkvIS8gEJxjw0w98CtkLmv8JQLzXHUuxSy9EVJy/Lc= X-Received: by 2002:ac8:4602:: with SMTP id p2mr10203695qtn.291.1567091328412; Thu, 29 Aug 2019 08:08:48 -0700 (PDT) MIME-Version: 1.0 References: <20190829114057.GZ71821@kib.kiev.ua> <28934eb780342605090bf365ac3a2e0d522256f5.camel@freebsd.org> In-Reply-To: <28934eb780342605090bf365ac3a2e0d522256f5.camel@freebsd.org> From: Warner Losh Date: Thu, 29 Aug 2019 09:08:37 -0600 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Ian Lepore Cc: Konstantin Belousov , Li-Wen Hsu , FreeBSD Hackers , fcp@freebsd.org X-Rspamd-Queue-Id: 46K5bj4qwcz3JZB X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=PgDmVkDk; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::833) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-5.91 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-0.996,0]; RCVD_IN_DNSWL_NONE(0.00)[3.3.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.91)[ip: (-9.33), ipnet: 2607:f8b0::/32(-2.85), asn: 15169(-2.32), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; FREEMAIL_CC(0.00)[gmail.com] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 15:08:50 -0000 On Thu, Aug 29, 2019 at 8:42 AM Ian Lepore wrote: > On Thu, 2019-08-29 at 14:40 +0300, Konstantin Belousov wrote: > > On Wed, Aug 28, 2019 at 12:29:58PM +0800, Li-Wen Hsu wrote: > > > It seems I was doing wrong that just changed the content of this FCP > > > to "feedback", but did not send to the right mailing lists. > > > > > > So I would like to make an announcement that the FCP > > > 20190401-ci_policy "CI policy": > > > > > > https://github.com/freebsd/fcp/blob/master/fcp-20190401-ci_policy.md > > > > > > is officially in "feedback" state to hopefully receive more comments > > > and suggestions, then we can move on for the next FCP state. > > > > What problem does the document tries to solve ? Or rather, do we really > > have the problem that it claims to solve ? > > > > From my experience, normal peer pressure is enough to get things fixed > > quickly when it is possible to fix them quickly. If there is something > > more non-trivial, esp. in the tests and not the build, I am sure that > > a rule allowing anybody to do blind revert is much more harmful than > > having a test broken. > > > > More, I know that tests are of very low quality, which means that > > brokeness of the tests is not an indicator of anything until root cause > > is identified. > > > > Can we rely on the common sense of developers until there is indeed the > > visible problem ? > > > > I totally agree. This is an overly-bureaucratic solution in search of > a problem. > > If this needs to be addressed at all (and I'm not sure it does), then > another sentence or two in bullet item 10 in section 18.1 [*] of the > committer's guide should be enough. And even then it needn't be > overly-formal and should just mention that if a commit does break the > build the committer is expected to be responsive to that problem and > the commit might get reverted if they're unresponsive. I don't think > we need schedules. (And I don't think breaking a test counts as > breaking the build.) > > [*] > https://www.freebsd.org/doc/en_US.ISO8859-1/articles/committers-guide/rules.html There's been growing friction around these points in the past several years. We should document that tree breakages aren't acceptable, committers are expected to fix things as soon as possible, and that new broken tests have to be investigated quickly, or the change should be reverted. This document tries to attach some suggested time frames on different types of breakage we've seen and set the expectations for people when things go wrong. Having different categories also allows us to set expectations for external toolchain breakage that's more lax than in-tree toolchain breakage, for example. The future will have external toolchain CI and notifications will likely be turned on when there's breakage. While some tests are poorly written, we should disable / remove the ones that are actually bad rather than create a policy that tolerates bad tests by setting no expectation around that. Now, one may quibble over the timelines, but that discussion helps everybody in the community know what the expectations are when committing and enables people that want to scrimp on testing to do so if they know they will be around for any unanticipated fallout, but may also encourage others to test more because they know they might not want to be. Warner From owner-freebsd-hackers@freebsd.org Thu Aug 29 15:09:02 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D22BED8B11; Thu, 29 Aug 2019 15:09:02 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-lj1-x243.google.com (mail-lj1-x243.google.com [IPv6:2a00:1450:4864:20::243]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46K5by52Vkz3JfT; Thu, 29 Aug 2019 15:09:02 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: by mail-lj1-x243.google.com with SMTP id l14so3415718lje.2; Thu, 29 Aug 2019 08:09:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:to:cc; bh=q+v6xLmI+5EfAk0WfFddMXmOZ8cUNN7J5HBg/AFEXdc=; b=ICOsuU4Mgn6hQRvyzUxmu9xnbFJxGaVLjg5kh8Z21ZZo0XB4j84kQ9++iJw0jk96Ai zRri8oVYQIC1/6ZSxuo9kPD4MP2cMiRnzFgUfX69K6DD4Cs7syrO8Wy9MmUfRd+WJS6v GjYqcr18ut+ZuH3/rjzOkbjhko/9ahkCkSNfaGq7WveSYmZtnVL7ZkzpsL+wvVAA5GZq Uu6agEWtSJzHFB74imQVwCiw1VCJ+UoNzJZRmr+1f6YM9VeDSBRYItKI7yeHmZ2H391m +m3Vb2siWiKMX/QvcwXugKFzfLtgg13vJZeBRby/PtF+BnqFZjiuz/sBMyl6tZnb8SCW ByWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=q+v6xLmI+5EfAk0WfFddMXmOZ8cUNN7J5HBg/AFEXdc=; b=GiNXpbB99NJ1BN2gafP7uTL0TLl8B+4BplopGJ1ioVb7Ga9V7/BT4YzNdGInG4iqv3 AmTbE9QJd2CpJYDvRcpzzhLuW5i1aaZP33uZAZFSKU+47Pqy4fjQ2tuwexQomQg5t9iw tykL6mmX5tJ4gU75jn50O/aY1najx2/kqXBgRbyGMvzOhcOHLvqW5JjW4prxdin6Z/EN 6QxFXs6KDX/kBsJlZYUnFOISPm5+kDNG/ufXQ2kjuFhx+2HRkDlfUn84X/PPfLbAS62X q341RREFG351sr3k3FtwMWtps4B6BDgeTyCeLmQWHrVKL+Hu+9SXcwx1+hEabzRV1Gio dLew== X-Gm-Message-State: APjAAAX+U/RbhZ7hQHNJcnLm8D6z3gmllS0OScwHPDn5rK1A52I1ziZF BjbIagXAo7Xy+Q12Qy9Yxr/HYcdqaZ0ZWh49Q+oub0iBuss= X-Google-Smtp-Source: APXvYqzgOgPvyT8Iruzjm0NRoATQ/vwYzJJVZAvZ+hu6Eb+DeduSjWQka4/IePMwIb72VKAhGaNQ41PiG79erNaR0Vc= X-Received: by 2002:a2e:81ca:: with SMTP id s10mr5721694ljg.181.1567091340970; Thu, 29 Aug 2019 08:09:00 -0700 (PDT) MIME-Version: 1.0 References: <20190829114057.GZ71821@kib.kiev.ua> <412537DD-D98F-4B92-85F5-CB93CF33F281@FreeBSD.org> <20190829144228.GA71821@kib.kiev.ua> In-Reply-To: Reply-To: araujo@freebsd.org From: Marcelo Araujo Date: Thu, 29 Aug 2019 23:08:49 +0800 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Kristof Provost Cc: Konstantin Belousov , FreeBSD Hackers , Li-Wen Hsu , fcp@freebsd.org X-Rspamd-Queue-Id: 46K5by52Vkz3JfT X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 15:09:02 -0000 Em qui, 29 de ago de 2019 =C3=A0s 23:03, Kristof Provost escreveu: > On 29 Aug 2019, at 16:42, Konstantin Belousov wrote: > > On Thu, Aug 29, 2019 at 02:03:00PM +0200, Kristof Provost wrote: > >> There are, somewhat regularly, commits which break functionality, or > >> at > >> the very least tests. > >> The main objective of this policy proposal is to try to improve > >> overall > >> code quality by encouraging and empowering all committers to > >> investigate > >> and fix test failures. > > But this policy does not encourage, if anything. > > It gives a free ticket to revert, discouraging committers. > > > To provide a counterpoint here: my personal frustration right now is > that I=E2=80=99ve spent a good bit of time adding tests for pf and fixing= bugs > for it, only to see the tests having to be disabled because of unrelated > (to pf) changes in the network stack. > > Either through lack of visibility, or lack of time, or because people > assume pf tests failures must by definition be the responsibility of the > pf maintainer, these failures have not been investigated by anyone other > than me, and I lack the time and subject matter expertise to fix them. > > I=E2=80=99m desperately afraid that if/when these bugs do get fixed we=E2= =80=99re > going to discover that other things have broken in the mean time, and > the tests are still going to fail, for different reasons. > > These are bugs. They=E2=80=99re the best case scenario for bug reports ev= en, > because they come with a reproduction case built-in, and yet they=E2=80= =99re > still not getting fixed. This too is discouraging. > > I=E2=80=99m open to alternative proposals for how to address that problem= , but > I don=E2=80=99t think that =E2=80=9Ccontinue on as we always have=E2=80= =9D is the correct > OK, because of PF that is sort of deprecated on FreeBSD and it need some new rules to make it workable, everybody else need to abdicate to some new rules. Yes, right you are!!!! > answer. > > Best regards, > Kristof > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > --=20 --=20 Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org \/ \ ^ Power To Server. .\. /_) From owner-freebsd-hackers@freebsd.org Thu Aug 29 15:09:34 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E8166D8C5E; Thu, 29 Aug 2019 15:09:34 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46K5cZ5Yt8z3Jpg; Thu, 29 Aug 2019 15:09:34 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.codepro.be", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id 76A2D1BFEA; Thu, 29 Aug 2019 15:09:34 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [10.0.2.193] (ptr-8rh08k12jlyqauo4swr.18120a2.ip6.access.telenet.be [IPv6:2a02:1811:240e:402:c59e:f85c:1ed8:db5b]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id EEAC138603; Thu, 29 Aug 2019 17:09:32 +0200 (CEST) From: "Kristof Provost" To: araujo@freebsd.org Cc: "Ian Lepore" , "Konstantin Belousov" , "FreeBSD Hackers" , "Li-Wen Hsu" , fcp@freebsd.org Subject: Re: FCP 20190401-ci_policy: CI policy Date: Thu, 29 Aug 2019 17:09:32 +0200 X-Mailer: MailMate (2.0BETAr6137) Message-ID: In-Reply-To: References: <20190829114057.GZ71821@kib.kiev.ua> <28934eb780342605090bf365ac3a2e0d522256f5.camel@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 15:09:35 -0000 On 29 Aug 2019, at 17:01, Marcelo Araujo wrote: > Em qui, 29 de ago de 2019 =C3=A0s 22:54, Kristof Provost > escreveu: > >> On 29 Aug 2019, at 16:42, Ian Lepore wrote: >>> (And I don't think breaking a test counts as >>> breaking the build.) >>> >> I fundamentally disagree on this point. A test failure is, just like = >> a >> compiler warning, a precious gift that should not be ignored. >> The more distance (both in terms of time, and in terms of the people >> involved) there is between a bug being introduced and it being = >> detected >> the harder it is to fix it. Test accelerate detection of bugs. If we = >> do >> not take test failures seriously (i.e. as an indication something is >> wrong and should be fixed) the tests will inevitable become useless = >> in >> one of two ways: we=E2=80=99ll either disable failing tests (which is = what = >> we >> tend to do now) reducing test coverage or we=E2=80=99ll have a test su= ite = >> with >> many failures in it, which makes it useless as well. (As with = >> compiler >> warnings, the best way to keep them under control is to consider them = >> to >> be fatal errors.) >> > > Could you elaborate where is the "fundamentally" you disagree? Where = > is the > fundament? You guys are introducing something new, yes everybody knows > about test, it is year 2019, but nobody can come with new rules tha in > hours we gonna revert if you "dare to don't fix it". Sorry, this is = > not how > people test software and fix it. > I do think that breaking a test breaks the build. Something used to work = and now it doesn=E2=80=99t. That=E2=80=99s breakage, even if it=E2=80=99s= not as total as = it not compiling any more. >> In either scenario we end up reducing test coverage, which means = >> we=E2=80=99re >> going to push more bugs towards users. >> >>> I totally agree. This is an overly-bureaucratic solution in search = >>> of >>> a problem. >>> >>> If this needs to be addressed at all (and I'm not sure it does), = >>> then >>> another sentence or two in bullet item 10 in section 18.1 [*] of the >>> committer's guide should be enough. And even then it needn't be >>> overly-formal and should just mention that if a commit does break = >>> the >>> build the committer is expected to be responsive to that problem and >>> the commit might get reverted if they're unresponsive. I don't = >>> think >>> we need schedules. >>> >> I do feel that=E2=80=99s a better argument. We=E2=80=99ve always had a= policy of >> reverting on request (AIUI), so this is more or less trying to be a >> strong restatement of that, more than a fundamental shift in policy. >> > > We don't have a policy to revert commit, actually revert commit is > something bad, it is kind of punishment, I have been there, nobody = > wants to > be there. Stop to push this non-sense argument. > That=E2=80=99s how I=E2=80=99ve interpreted =E2=80=9911. Developer Relati= ons=E2=80=99 in in = https://www.freebsd.org/doc/en_US.ISO8859-1/articles/committers-guide/art= icle.html#conventions = (Specifically, =E2=80=9C If a commit does results in controversy erupti= ng, = it may be advisable to consider backing the change out again until the = matter is settled.=E2=80=9D) I understand that it=E2=80=99s not fun to see changes reverted, and it=E2= =80=99s = certainly not the intention to make that the preferred solution. = That=E2=80=99s why the FCP discusses adds waiting periods and discussion = with = the committer. Best regards, Kristof From owner-freebsd-hackers@freebsd.org Thu Aug 29 15:10:06 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 75316D8DB0 for ; Thu, 29 Aug 2019 15:10:06 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x731.google.com (mail-qk1-x731.google.com [IPv6:2607:f8b0:4864:20::731]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46K5d95JFZz3Jy6 for ; Thu, 29 Aug 2019 15:10:05 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x731.google.com with SMTP id f13so3165037qkm.9 for ; Thu, 29 Aug 2019 08:10:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=hPOjkvVLjUDz8PTEAgwVogrcGUwECCcytgUPPKENR04=; b=z7elCk/3resJpmTxkLZZf/zVFFXNsIcyCk03541nM4tV7nETNaysrc+u6ppfX2M/3B j7GR5LIaGBjunYT9o6PmjkZNQAhrNVus59xeU3J1WVWP9FetlRosPVCQdRIBCq545V2c LqhjIjTjF96xj8p1x5JLXW8RkJpsWOVZRS+Uqmnw21e23JbEOTrLtoM1tQgs42iz5mnO I/Hc6/UvePsJZE7fjyI4C65xkI0az6Ud7eFhgsisqW0dVxBgN17en537uBY6+VlqostM 9GhTMCq8rqZR6mUvTVk/tZjkNymuq7U8AeEI7EtS5cnv/El9Tnn+jxYhuV2TFfRTqiZg 05TQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=hPOjkvVLjUDz8PTEAgwVogrcGUwECCcytgUPPKENR04=; b=Mq2P1pMXoMSipGy+lCi/pbY5u+udVUXGVMUgOf/TY8Xe2A0urSfKa/n5Z0lOCFZHEM 62JKitN/Vu8/mgrP4z7WhtQw6bJQXtFegULIwUhpoFwoslmUiVpvBdhs/bWZxYdoqtAF ZKATrUwn6ygnmiaWicm/hvPWEmH7ScCulKHuF8GuIROqGDBOPlu4MmABvtx3drst+W4y R2hzoy1FUzgt+BS5NNesIYx39JNhQmA+75zWpDy15q1mIOybN82w5OPzet8qE9eqOZ99 AtbndYO8tRjGrp1wI3nG6MrKP1LZ/dDlBQeXRL6KYd0Gniq11uxKWHbq7RP/aK+K6P64 LCZg== X-Gm-Message-State: APjAAAXQyoUl+nuf2vFbKg+TyHccYZHYv6yhA2TJWHjl7i3t9Xxr0nui dxcPbSnbUj3DHCwBFvpDVmWnhKyu9Wg+xUEWFKuXPQ== X-Google-Smtp-Source: APXvYqwFgJdDNLwBgKx77ozvLQ5s+GCLcQKQljZwWMUdZB+rpDKRK614tXrK+5zJwsjFkYCvzB0rXWTlo9yArBPkZC4= X-Received: by 2002:a37:2c07:: with SMTP id s7mr9818010qkh.495.1567091404529; Thu, 29 Aug 2019 08:10:04 -0700 (PDT) MIME-Version: 1.0 References: <20190829114057.GZ71821@kib.kiev.ua> <412537DD-D98F-4B92-85F5-CB93CF33F281@FreeBSD.org> <20190829144228.GA71821@kib.kiev.ua> In-Reply-To: From: Warner Losh Date: Thu, 29 Aug 2019 09:09:53 -0600 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Marcelo Araujo Cc: Kristof Provost , Konstantin Belousov , FreeBSD Hackers , Li-Wen Hsu , fcp@freebsd.org X-Rspamd-Queue-Id: 46K5d95JFZz3Jy6 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=z7elCk/3; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::731) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-5.91 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-0.996,0]; RCVD_IN_DNSWL_NONE(0.00)[1.3.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.91)[ip: (-9.35), ipnet: 2607:f8b0::/32(-2.85), asn: 15169(-2.32), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 15:10:06 -0000 On Thu, Aug 29, 2019 at 9:09 AM Marcelo Araujo wrote: > Em qui, 29 de ago de 2019 =C3=A0s 23:03, Kristof Provost > escreveu: > > > On 29 Aug 2019, at 16:42, Konstantin Belousov wrote: > > > On Thu, Aug 29, 2019 at 02:03:00PM +0200, Kristof Provost wrote: > > >> There are, somewhat regularly, commits which break functionality, or > > >> at > > >> the very least tests. > > >> The main objective of this policy proposal is to try to improve > > >> overall > > >> code quality by encouraging and empowering all committers to > > >> investigate > > >> and fix test failures. > > > But this policy does not encourage, if anything. > > > It gives a free ticket to revert, discouraging committers. > > > > > To provide a counterpoint here: my personal frustration right now is > > that I=E2=80=99ve spent a good bit of time adding tests for pf and fixi= ng bugs > > for it, only to see the tests having to be disabled because of unrelate= d > > (to pf) changes in the network stack. > > > > Either through lack of visibility, or lack of time, or because people > > assume pf tests failures must by definition be the responsibility of th= e > > pf maintainer, these failures have not been investigated by anyone othe= r > > than me, and I lack the time and subject matter expertise to fix them. > > > > I=E2=80=99m desperately afraid that if/when these bugs do get fixed we= =E2=80=99re > > going to discover that other things have broken in the mean time, and > > the tests are still going to fail, for different reasons. > > > > These are bugs. They=E2=80=99re the best case scenario for bug reports = even, > > because they come with a reproduction case built-in, and yet they=E2=80= =99re > > still not getting fixed. This too is discouraging. > > > > I=E2=80=99m open to alternative proposals for how to address that probl= em, but > > I don=E2=80=99t think that =E2=80=9Ccontinue on as we always have=E2=80= =9D is the correct > > > > OK, because of PF that is sort of deprecated on FreeBSD and it need some > new rules to make it workable, everybody else need to abdicate to some ne= w > rules. Yes, right you are!!!! > Let's take every opportunity to clarify community norms and turn it into a federal case. That's productive. Warner From owner-freebsd-hackers@freebsd.org Thu Aug 29 15:13:07 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4E772D9120; Thu, 29 Aug 2019 15:13:07 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com [IPv6:2a00:1450:4864:20::132]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46K5hf10pQz3KRk; Thu, 29 Aug 2019 15:13:05 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: by mail-lf1-x132.google.com with SMTP id r134so1925338lff.12; Thu, 29 Aug 2019 08:13:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:to:cc; bh=h0QMz+Lm8FEUOwvzCgeLbqnlJx+2a0mAxRB2YpPElmc=; b=oeml8Be48kyt6U9nTOFc6KyjbIdOcnb3Q+4/ZzNCUn3tbB5g7n7qrKhGrPD21RqJEv Bo+EdopkKyhIR6Gt0u1wiJF+AkzHVZ56/7rY8z9svqj6YXyA4DpdB+YNFusLD8cWf8sJ Wf0YArIZ1tG0mNOKHH53j05ICTlqQkNc/3IKZ/5fHoqFcJrnHJte6Wf+bNi7K2gMSjFt cugSXQX0/4RWEK6Y/AApgynYB2SQG5D9HDOCgfLO2QX0jQTsCxZ7+paZDd2WU5Jb6zre nUv5sQ/mo5czl8hMwl0cNtxUieAaFBq9Jey4QhxQr1p0UWq+7k8HIUPHWTLtHyYfWfwc o6VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=h0QMz+Lm8FEUOwvzCgeLbqnlJx+2a0mAxRB2YpPElmc=; b=n+zW64dil5giiguJ/4THzlLPAB2YRq5xl7UIAcKLXNgr11d42Fuv4HriBbBXXjfSwM LQ78n6vDprJ84iUe2b7gyKrArIouqY/9tU0nnc/3m20vH3VKuL0sHs66rofaQydcud0z GxDhKk+XogMffN2QPvEUh6CXDvTw0ZB/0GnFGFsYgsuPFF3vmlIYSGthD3TMPtisXXnI S3yS6pO1MGmyJjBIERnXWvHUzuw166KdzfaLZ27qx2/vB25X8fa3uTlVVpAYopHH/tE+ UjT6aCIN6RdKbRgdQI+9Y6qUOLEwjio0XbNUMXTIdlG+pDOtIHMXVll0stXB/9XfkLLj OFxQ== X-Gm-Message-State: APjAAAVMaNd3mr5bgPZZDW2TvUb8n3GTEqAaLtphlNFZhGBv2BSyOffe pT3qZCINYBsa9MOhmweAdA1Qa1RRVRnGHYM0JJc= X-Google-Smtp-Source: APXvYqyBoaxALyIyxqXLD1UbhXziGmNMbQfbPUoEla3gz2Djnd5WeIp9kPMzsG7nr1DoZrK+pWwGtz+vZZ9x0L2n784= X-Received: by 2002:ac2:5485:: with SMTP id t5mr4501980lfk.27.1567091582825; Thu, 29 Aug 2019 08:13:02 -0700 (PDT) MIME-Version: 1.0 References: <20190829114057.GZ71821@kib.kiev.ua> <412537DD-D98F-4B92-85F5-CB93CF33F281@FreeBSD.org> <20190829144228.GA71821@kib.kiev.ua> In-Reply-To: Reply-To: araujo@freebsd.org From: Marcelo Araujo Date: Thu, 29 Aug 2019 23:12:51 +0800 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Warner Losh Cc: Marcelo Araujo , Kristof Provost , Konstantin Belousov , FreeBSD Hackers , Li-Wen Hsu , fcp@freebsd.org X-Rspamd-Queue-Id: 46K5hf10pQz3KRk X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=oeml8Be4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of araujobsdport@gmail.com designates 2a00:1450:4864:20::132 as permitted sender) smtp.mailfrom=araujobsdport@gmail.com X-Spamd-Result: default: False [-3.99 / 15.00]; HAS_REPLYTO(0.00)[araujo@freebsd.org]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCPT_COUNT_SEVEN(0.00)[7]; NEURAL_HAM_SHORT(-0.99)[-0.993,0]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (-9.54), ipnet: 2a00:1450::/32(-2.99), asn: 15169(-2.32), country: US(-0.05)]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 15:13:07 -0000 Em qui, 29 de ago de 2019 =C3=A0s 23:10, Warner Losh escre= veu: > > > On Thu, Aug 29, 2019 at 9:09 AM Marcelo Araujo > wrote: > >> Em qui, 29 de ago de 2019 =C3=A0s 23:03, Kristof Provost >> escreveu: >> >> > On 29 Aug 2019, at 16:42, Konstantin Belousov wrote: >> > > On Thu, Aug 29, 2019 at 02:03:00PM +0200, Kristof Provost wrote: >> > >> There are, somewhat regularly, commits which break functionality, o= r >> > >> at >> > >> the very least tests. >> > >> The main objective of this policy proposal is to try to improve >> > >> overall >> > >> code quality by encouraging and empowering all committers to >> > >> investigate >> > >> and fix test failures. >> > > But this policy does not encourage, if anything. >> > > It gives a free ticket to revert, discouraging committers. >> > > >> > To provide a counterpoint here: my personal frustration right now is >> > that I=E2=80=99ve spent a good bit of time adding tests for pf and fix= ing bugs >> > for it, only to see the tests having to be disabled because of unrelat= ed >> > (to pf) changes in the network stack. >> > >> > Either through lack of visibility, or lack of time, or because people >> > assume pf tests failures must by definition be the responsibility of t= he >> > pf maintainer, these failures have not been investigated by anyone oth= er >> > than me, and I lack the time and subject matter expertise to fix them. >> > >> > I=E2=80=99m desperately afraid that if/when these bugs do get fixed we= =E2=80=99re >> > going to discover that other things have broken in the mean time, and >> > the tests are still going to fail, for different reasons. >> > >> > These are bugs. They=E2=80=99re the best case scenario for bug reports= even, >> > because they come with a reproduction case built-in, and yet they=E2= =80=99re >> > still not getting fixed. This too is discouraging. >> > >> > I=E2=80=99m open to alternative proposals for how to address that prob= lem, but >> > I don=E2=80=99t think that =E2=80=9Ccontinue on as we always have=E2= =80=9D is the correct >> > >> >> OK, because of PF that is sort of deprecated on FreeBSD and it need some >> new rules to make it workable, everybody else need to abdicate to some n= ew >> rules. Yes, right you are!!!! >> > > Let's take every opportunity to clarify community norms and turn it into = a > federal case. That's productive. > Yeah, that was my bad!!! Apologies for that if we still have time. Sorry for that. > > Warner > --=20 --=20 Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org \/ \ ^ Power To Server. .\. /_) From owner-freebsd-hackers@freebsd.org Thu Aug 29 15:37:20 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C6052D9C9A; Thu, 29 Aug 2019 15:37:20 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46K6Dc3xYNz3Lwf; Thu, 29 Aug 2019 15:37:20 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x7TFbChh018671 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 29 Aug 2019 18:37:15 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x7TFbChh018671 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x7TFbCjH018670; Thu, 29 Aug 2019 18:37:12 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 29 Aug 2019 18:37:12 +0300 From: Konstantin Belousov To: Kristof Provost Cc: Li-Wen Hsu , FreeBSD Hackers , fcp@freebsd.org Subject: Re: FCP 20190401-ci_policy: CI policy Message-ID: <20190829153712.GB71821@kib.kiev.ua> References: <20190829114057.GZ71821@kib.kiev.ua> <412537DD-D98F-4B92-85F5-CB93CF33F281@FreeBSD.org> <20190829144228.GA71821@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-Rspamd-Queue-Id: 46K6Dc3xYNz3Lwf X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.90 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.91)[-0.906,0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 15:37:20 -0000 On Thu, Aug 29, 2019 at 05:02:47PM +0200, Kristof Provost wrote: > On 29 Aug 2019, at 16:42, Konstantin Belousov wrote: > > On Thu, Aug 29, 2019 at 02:03:00PM +0200, Kristof Provost wrote: > >> There are, somewhat regularly, commits which break functionality, or > >> at > >> the very least tests. > >> The main objective of this policy proposal is to try to improve > >> overall > >> code quality by encouraging and empowering all committers to > >> investigate > >> and fix test failures. > > But this policy does not encourage, if anything. > > It gives a free ticket to revert, discouraging committers. > > > To provide a counterpoint here: my personal frustration right now is > that I’ve spent a good bit of time adding tests for pf and fixing bugs > for it, only to see the tests having to be disabled because of unrelated > (to pf) changes in the network stack. > > Either through lack of visibility, or lack of time, or because people > assume pf tests failures must by definition be the responsibility of the > pf maintainer, these failures have not been investigated by anyone other > than me, and I lack the time and subject matter expertise to fix them. > > I’m desperately afraid that if/when these bugs do get fixed we’re > going to discover that other things have broken in the mean time, and > the tests are still going to fail, for different reasons. > > These are bugs. They’re the best case scenario for bug reports even, > because they come with a reproduction case built-in, and yet they’re > still not getting fixed. This too is discouraging. I fully agree with your attitude there, and understand your frustration. IMO the right action would be to contact the committers who did the relevant changes, first. Was it done ? What was their response ? If they are silent, next action would be some public mail. Do you know where the bug is ? If yes, how hard is to fix it ? > > I’m open to alternative proposals for how to address that problem, but > I don’t think that “continue on as we always have” is the correct > answer. > > Best regards, > Kristof From owner-freebsd-hackers@freebsd.org Thu Aug 29 15:52:46 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A090BDA481; Thu, 29 Aug 2019 15:52:46 +0000 (UTC) (envelope-from lwhsu.freebsd@gmail.com) Received: from mail-yw1-f48.google.com (mail-yw1-f48.google.com [209.85.161.48]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46K6ZP5rFhz3N0J; Thu, 29 Aug 2019 15:52:45 +0000 (UTC) (envelope-from lwhsu.freebsd@gmail.com) Received: by mail-yw1-f48.google.com with SMTP id i207so1296530ywc.9; Thu, 29 Aug 2019 08:52:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=wY3QqHOPLSUlCOb4QmA9woMOMytMceJZ9WNhi9JsvlM=; b=CYWnJ8vNpdiBVrHFHRT05p1bEF2BUGCKrxwembIDbWQtYG4SjLr/cYSHStplFMSU9c ySjsDcJpvm1Ev9lK8ROgBOwvoD8opCnLq+y3IfYIy0mqhdcWDmm0cY2zM+1TVFBE3Arl 4IgfBWCIv13a/NMLFK8TNwF3L4OJGhzLEYF60Qwk4sdT7oI3GT9SAmf1MCpiJO8w+vih a3E18wLVVx9ux2XAz0RMWIlApTLgLBvZlXmsFo+R9NIpsgsLJPXPU8mcuvcTWtr4GVhm lvDLFRSd/nt7D+7YKl8maTtsQbPMMPUU7kat2UDuwRjvaVEfg1r5jA4NnWmQCzCEpS+c 9c2A== X-Gm-Message-State: APjAAAUE9JMFS0i/ncc/76YqR4tgkmV3LDibGm51zzVDvPphKlMqU+j+ 4mFdzKJ7V4TUxHDuQV5fMlTbY3b9sz918vh75UN1FeV9 X-Google-Smtp-Source: APXvYqwSf91pWiPN9dYEGQSlr8I/x8+1FRo41YZhZ6mp+Ir88nYMezpj5tvO48mwXFlTygEOH5/EFE6wnhk3eQUzpmk= X-Received: by 2002:a0d:ca02:: with SMTP id m2mr7784840ywd.400.1567093964422; Thu, 29 Aug 2019 08:52:44 -0700 (PDT) MIME-Version: 1.0 References: <20190829114057.GZ71821@kib.kiev.ua> <412537DD-D98F-4B92-85F5-CB93CF33F281@FreeBSD.org> <20190829144228.GA71821@kib.kiev.ua> <20190829153712.GB71821@kib.kiev.ua> In-Reply-To: <20190829153712.GB71821@kib.kiev.ua> From: Li-Wen Hsu Date: Thu, 29 Aug 2019 23:52:29 +0800 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: FreeBSD Hackers , fcp@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46K6ZP5rFhz3N0J X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of lwhsufreebsd@gmail.com designates 209.85.161.48 as permitted sender) smtp.mailfrom=lwhsufreebsd@gmail.com X-Spamd-Result: default: False [-5.93 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; TAGGED_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_TLS_ALL(0.00)[]; IP_SCORE(-2.93)[ip: (-8.95), ipnet: 209.85.128.0/17(-3.34), asn: 15169(-2.32), country: US(-0.05)]; NEURAL_HAM_SHORT(-0.99)[-0.994,0]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[48.161.85.209.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[lwhsu@freebsd.org,lwhsufreebsd@gmail.com]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[lwhsu@freebsd.org,lwhsufreebsd@gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 15:52:46 -0000 I was really hesitated to send out the previous mail because I understand this is a sensitive topic and some words might look scary. My apologies if the proposal makes people uncomfortable. Please calm down, this is just an idea, not call for vote or say it will proceed as-is. We want to point out there is a thing may worth to be discussed, and hope to collect the comments and suggestions, we know the proposed way is not perfect, and that's why we need better idea from more people. I am really bad at writing, but some items I hope people can check: - Yes we're doing good now, and we should keep it and let's try to do better. - Please check the weekly CI report to see the build (and test) statistics, we still have some rooms to be improved. Let's try to get bugs fixed before users asking on -current or -stable. Of course we cannot cover all parts of the system, but that means we can always do more contributions. - We have been running CI for few years, perhaps it's still too early, but let's try to pay more attention to it. If there are things imperfect, let's fix it. I cannot watch the results, do preliminary analysis and call people to check, just by myself along, at least not forever. I hope we can have a more automatic and scalable way. - There is no one want to "control" the contribution, instead, we hope this could make collaboration more smoothly. As there are more and more contributors, we should have a way to keep head and stable branches buildable and have less regressions as we can as possible, then everyone can work together with more confidence. - The "revert" part and the timeline looks scary, please think this is the last and unwanted solution. The description is try to limit its scope and encourage people do analysis, communication and fix first. The words here absolutely should be improved. Updating the committer guide is really a good idea, and we probably should define what means "unresponsive", like in ports we have maintainer timeout and even maintainer reset. Thanks the feedback from all of you. I think encouraging people to discuss and keep the discussion record is one of the purposes to have FCP process. Best, Li-Wen From owner-freebsd-hackers@freebsd.org Thu Aug 29 19:05:14 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3E1C4DDABC; Thu, 29 Aug 2019 19:05:14 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46KBrT2SJ5z44l2; Thu, 29 Aug 2019 19:05:12 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x7TJ5BRs091372; Thu, 29 Aug 2019 12:05:11 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x7TJ5Bw8091371; Thu, 29 Aug 2019 12:05:11 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201908291905.x7TJ5Bw8091371@gndrsh.dnsmgr.net> Subject: Re: FCP 20190401-ci_policy: CI policy In-Reply-To: To: araujo@freebsd.org Date: Thu, 29 Aug 2019 12:05:11 -0700 (PDT) CC: Kristof Provost , Konstantin Belousov , FreeBSD Hackers , Li-Wen Hsu , Ian Lepore , fcp@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 46KBrT2SJ5z44l2 X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of freebsd-rwg@gndrsh.dnsmgr.net has no SPF policy when checking 69.59.192.140) smtp.mailfrom=freebsd-rwg@gndrsh.dnsmgr.net X-Spamd-Result: default: False [2.65 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.49)[0.485,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[dnsmgr.net]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.95)[0.946,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.28)[0.278,0]; RCPT_COUNT_SEVEN(0.00)[7]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.04)[ip: (0.15), ipnet: 69.59.192.0/19(0.07), asn: 13868(0.05), country: US(-0.05)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 19:05:14 -0000 (unneeded context removed) > > In either scenario we end up reducing test coverage, which means we?re > > going to push more bugs towards users. > > > > > I totally agree. This is an overly-bureaucratic solution in search of > > > a problem. > > > > > > If this needs to be addressed at all (and I'm not sure it does), then > > > another sentence or two in bullet item 10 in section 18.1 [*] of the > > > committer's guide should be enough. And even then it needn't be > > > overly-formal and should just mention that if a commit does break the > > > build the committer is expected to be responsive to that problem and > > > the commit might get reverted if they're unresponsive. I don't think > > > we need schedules. > > > > > I do feel that?s a better argument. We?ve always had a policy of > > reverting on request (AIUI), so this is more or less trying to be a > > strong restatement of that, more than a fundamental shift in policy. > > > > We don't have a policy to revert commit, actually revert commit is > something bad, it is kind of punishment, I have been there, nobody wants to > be there. Stop to push this non-sense argument. Here in lies one of the fundemental problems, this view by some that a "revert commit is something bad, it is kind of punishment". That is not true. Reverts are GREAT things, they allow the tree to be returned to a known state, usually quicly. The original commit is STILL IN SVN, and a bad revert can guess what.. be reverted!. IMHO the project as a whole needs to overcome its fear of reverts and start to use them for the great and powerful things that they are. This connection of bad and punishment needs to stop, and the sooner the better. -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-hackers@freebsd.org Thu Aug 29 21:26:30 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E2BBBE04E5 for ; Thu, 29 Aug 2019 21:26:30 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x72d.google.com (mail-qk1-x72d.google.com [IPv6:2607:f8b0:4864:20::72d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KFzT6yr1z4CtY for ; Thu, 29 Aug 2019 21:26:29 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x72d.google.com with SMTP id m2so4318588qkd.10 for ; Thu, 29 Aug 2019 14:26:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=psQkg5NFhz4GN6E0Jq/7T4mBDwPB1ph0P/vT6e2lXuQ=; b=JZd6loCg+YQdQql8wgOiMWtOt5vOp+II5onQh8aReWkG7MvshujPEnEh9AJ54oKpFk ByuiRvoyLjTSkE4FiGYIUIq4P34CkF/y9IV1qDX7DC/DnnSvI91/X+IcW6j55wR8iyRx rJXROucxRC0RKNODzMr1G73wp0F/xgHEtM7GS7XmwS/kvLvzcfxeqL5mEGBQDPqik+nt mlhNyUmBl1CtiA5e3Y6kHABkKBiCt3lGjenwHDMutQRi+nI3Cul9buFSDGEaOzckEQXN xCRXbc/AOd+OEDe8ZtaVUBd0FwkwoWgCsn7/aocVL6Ri8WwOWI9MTEemQfAzSdq5fue4 tleg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=psQkg5NFhz4GN6E0Jq/7T4mBDwPB1ph0P/vT6e2lXuQ=; b=q1bBF4KOR5wxLix+9Wkrn1cBjOulaC2074Jb34GzbqpknV0js7vO4McEUqIcmxa/jN OqtUCuikft3DzVZfymVwU3OW0/X/Qfu0nTNTuHmcj9LRHuyCE40kI/3fOxSvWXavEHBg wnjJJCbltKGciK6Mv8OzYjoEnzdQorRUqqQzC3H5wwSzZP1Zw9oy5wYezZWmDhVD/PEv eNBc3Fh+q+NQrqORSitTkwaaOdS6V5MbeunhmGv1HTEWXSKaLZDzwcURaTQ5VVucxVAa kIAemuw59t784zUCq0BwS+k4bv5I/Kw4dMpBXFjLIXrl+w/M7bcNF7b0/BBqAShP+72X MTDw== X-Gm-Message-State: APjAAAU6bCkmWDRPVNGiPZGe2qr0Y5bVr+FaDr7LYEQM+/Qb/SRJFxb6 TlcYQVfEla/1SvY9jBjOnzpFzjhLx9DzRgDmaII8Mw== X-Google-Smtp-Source: APXvYqxPX6vTEYq6x5eAL4kdMewdUKcE6kOLkS1/XnuFVPYFqgjiLIDWu/r+la/Te+iNG3KoxtoAmIP0CU7xocCGeSQ= X-Received: by 2002:a37:4b03:: with SMTP id y3mr12266242qka.215.1567113988934; Thu, 29 Aug 2019 14:26:28 -0700 (PDT) MIME-Version: 1.0 References: <201908291905.x7TJ5Bw8091371@gndrsh.dnsmgr.net> In-Reply-To: <201908291905.x7TJ5Bw8091371@gndrsh.dnsmgr.net> From: Warner Losh Date: Thu, 29 Aug 2019 15:26:17 -0600 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: "Rodney W. Grimes" Cc: Marcelo Araujo , Ian Lepore , FreeBSD Hackers , fcp@freebsd.org, Konstantin Belousov , Li-Wen Hsu , Kristof Provost X-Rspamd-Queue-Id: 46KFzT6yr1z4CtY X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=JZd6loCg; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::72d) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-4.91 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; URI_COUNT_ODD(1.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-0.99)[-0.991,0]; RCVD_IN_DNSWL_NONE(0.00)[d.2.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCPT_COUNT_SEVEN(0.00)[8]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.92)[ip: (-9.38), ipnet: 2607:f8b0::/32(-2.85), asn: 15169(-2.32), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 21:26:30 -0000 On Thu, Aug 29, 2019 at 1:05 PM Rodney W. Grimes < freebsd-rwg@gndrsh.dnsmgr.net> wrote: > (unneeded context removed) > > > > In either scenario we end up reducing test coverage, which means we?re > > > going to push more bugs towards users. > > > > > > > I totally agree. This is an overly-bureaucratic solution in search > of > > > > a problem. > > > > > > > > If this needs to be addressed at all (and I'm not sure it does), then > > > > another sentence or two in bullet item 10 in section 18.1 [*] of the > > > > committer's guide should be enough. And even then it needn't be > > > > overly-formal and should just mention that if a commit does break the > > > > build the committer is expected to be responsive to that problem and > > > > the commit might get reverted if they're unresponsive. I don't think > > > > we need schedules. > > > > > > > I do feel that?s a better argument. We?ve always had a policy of > > > reverting on request (AIUI), so this is more or less trying to be a > > > strong restatement of that, more than a fundamental shift in policy. > > > > > > > We don't have a policy to revert commit, actually revert commit is > > something bad, it is kind of punishment, I have been there, nobody wants > to > > be there. Stop to push this non-sense argument. > > Here in lies one of the fundemental problems, this view by some that > a "revert commit is something bad, it is kind of punishment". That is > not true. Reverts are GREAT things, they allow the tree to be returned > to a known state, usually quicly. The original commit is STILL IN SVN, > and a bad revert can guess what.. be reverted!. > > IMHO the project as a whole needs to overcome its fear of reverts and > start to use them for the great and powerful things that they are. > > This connection of bad and punishment needs to stop, and the sooner > the better. > > -- > Rod Grimes > rgrimes@freebsd.org > _______________________________________________ > freebsd-fcp@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fcp > To unsubscribe, send any mail to "freebsd-fcp-unsubscribe@freebsd.org" > From owner-freebsd-hackers@freebsd.org Thu Aug 29 21:32:30 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4182FE07E6 for ; Thu, 29 Aug 2019 21:32:30 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x82e.google.com (mail-qt1-x82e.google.com [IPv6:2607:f8b0:4864:20::82e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KG6P37kvz4DNG for ; Thu, 29 Aug 2019 21:32:29 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x82e.google.com with SMTP id r5so169627qtd.0 for ; Thu, 29 Aug 2019 14:32:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:from:date:message-id:subject:to:cc; bh=t370hma+L6lmoL29ph5rMXaIN65CYesVtRZQtj6/L3Y=; b=2OeaZ8GYhg7L52BtAAYIKbGd4620qyJXxpLw8+CfVkTuLeVACLc8skdM4MF0SqOvb3 o2L6vDWdvwaVymd5TkKDjGARfZTGWQtThdChCDfzqqDyC663SS69EwIIHB2ht/ia9YHO 7mE1K2dgAspXb4pZKx+FJm7I41fTH4nzRREOuGGlY7nVe3x+tJBWHDYo4pqhNwJNXqej tHJhTEhvDiWejEYynxzHEYxcuVpfKtD8gR5OGz0KeQYf0D8iG/OxeqLGp87zn3pIIzgX nO65JULsffWqCZ5QYtLeCM2Mj7UYqhmhK8o/RAarqJfFw6JKO+ypWqZFtuxlaZrS1Ban oErw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:from:date:message-id :subject:to:cc; bh=t370hma+L6lmoL29ph5rMXaIN65CYesVtRZQtj6/L3Y=; b=uEiiwUfiuRILbPFGu6flDh+hPctV/ydkZNsWVL0WdqkKy+KiLFBlqt1DgIHcnJ4mlE gFBrbxRUi6uWMClmTUvuhVVdE7/xFV+y7yu51tGwLHdMp+3yBnwcrAOthiO/33wv/bzQ sWUSL1bMyJTJWYHGNmiJpY6VzM5tjIp6wqcSox4V6arNgmO9N+6rHO1xD/vOAar4ZdAX skoR2scfJpntH+A2vsnjVIBX6sPuMxorrMGo5KeMot8zbHd5PDNqBLKk2lB3jJ4AX0+N SQgUzDMgo95ea5+ct36fq+MRJ01Sd1xn8HYQm4AMibo6tzeZTvWG3A7czHI0gjOAx+U3 OImQ== X-Gm-Message-State: APjAAAXrU0m5xcaDs9G9an49gxiMLZ8sCab5R1RQz+ksIGIVtI5KZA+W 9g8hHq923UpNDAaGIr/ROpk5vJBmQR6dEyeY+iNjig== X-Google-Smtp-Source: APXvYqzu/TEaMKEGJaicoo/kG89YsEyHSSBwdn/EWwXCDFL8TNWn9IHFu4aXk465mkEFyOIo19ZHinj31xCclBp5N7g= X-Received: by 2002:a0c:f6c6:: with SMTP id d6mr8235136qvo.102.1567114348210; Thu, 29 Aug 2019 14:32:28 -0700 (PDT) MIME-Version: 1.0 References: <201908291905.x7TJ5Bw8091371@gndrsh.dnsmgr.net> From: Warner Losh Date: Thu, 29 Aug 2019 15:32:17 -0600 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: "Rodney W. Grimes" Cc: Marcelo Araujo , Ian Lepore , FreeBSD Hackers , fcp@freebsd.org, Konstantin Belousov , Li-Wen Hsu , Kristof Provost X-Rspamd-Queue-Id: 46KG6P37kvz4DNG X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=2OeaZ8GY; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::82e) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-5.90 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-0.996,0]; RCVD_IN_DNSWL_NONE(0.00)[e.2.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCPT_COUNT_SEVEN(0.00)[8]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.91)[ip: (-9.32), ipnet: 2607:f8b0::/32(-2.84), asn: 15169(-2.32), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 21:32:30 -0000 On Thu, Aug 29, 2019 at 3:26 PM Warner Losh wrote: > > > On Thu, Aug 29, 2019 at 1:05 PM Rodney W. Grimes < > freebsd-rwg@gndrsh.dnsmgr.net> wrote: > >> (unneeded context removed) >> >> > > In either scenario we end up reducing test coverage, which means we?re >> > > going to push more bugs towards users. >> > > >> > > > I totally agree. This is an overly-bureaucratic solution in search >> of >> > > > a problem. >> > > > >> > > > If this needs to be addressed at all (and I'm not sure it does), >> then >> > > > another sentence or two in bullet item 10 in section 18.1 [*] of the >> > > > committer's guide should be enough. And even then it needn't be >> > > > overly-formal and should just mention that if a commit does break >> the >> > > > build the committer is expected to be responsive to that problem and >> > > > the commit might get reverted if they're unresponsive. I don't >> think >> > > > we need schedules. >> > > > >> > > I do feel that?s a better argument. We?ve always had a policy of >> > > reverting on request (AIUI), so this is more or less trying to be a >> > > strong restatement of that, more than a fundamental shift in policy. >> > > >> > >> > We don't have a policy to revert commit, actually revert commit is >> > something bad, it is kind of punishment, I have been there, nobody >> wants to >> > be there. Stop to push this non-sense argument. >> >> Here in lies one of the fundemental problems, this view by some that >> a "revert commit is something bad, it is kind of punishment". That is >> not true. Reverts are GREAT things, they allow the tree to be returned >> to a known state, usually quicly. The original commit is STILL IN SVN, >> and a bad revert can guess what.. be reverted!. >> >> IMHO the project as a whole needs to overcome its fear of reverts and >> start to use them for the great and powerful things that they are. >> >> This connection of bad and punishment needs to stop, and the sooner >> the better. >> > In the past, if someone had any follow on work at all in their tree, the reversion would be quite disruptive to that work. Most of the time it's a lot easier for me, with a lot less friction, to just fix issues that come up after the commit than to revert and prepare a new commit. Sure, it's possible, but it can destroy work in extreme cases. *THAT* is why I'm firmly in the camp of giving the original committer a shot at fixing things because it's much less disruptive to them, and generally we can get a fix into the tree faster. It reduces friction and encourages people to fix things quickly, imho, to hesitate a little on the revert. Especailly when the broken thing is the playstation loader on powerpc that can stay broken for the hour or six (or even days) it takes me to figure out why it broke... Often things away from the beaten path don't get discovered for days or weeks or months, and a reversion then can be extremely disruptive if there's other changes layered on top of the offending commit.... So the whole reversion issue is a lot more complicated than 'oh, it's still in svn'. There are real high costs associated with being too quick or liberal on the revert and those must be weighed against the damage the bad commit is doing.. Warner From owner-freebsd-hackers@freebsd.org Thu Aug 29 23:27:21 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 002E5E260A; Thu, 29 Aug 2019 23:27:21 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-io1-f50.google.com (mail-io1-f50.google.com [209.85.166.50]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KJfw1NcJz4JdM; Thu, 29 Aug 2019 23:27:19 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-io1-f50.google.com with SMTP id d25so7881750iob.6; Thu, 29 Aug 2019 16:27:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=dEoLdEVaF4XHeFHQy6IytzlYnAzvIf+XzwOID05rIfA=; b=bjwIB44oeNBMH3yjASjJOu8URI39+yfVaWR1SsIwmWEn1nUw182IkP4ewOzu2HRVB9 BIsq4jK5w1Xw4Vc7bPQacrQMBay6ZrhMmbxK+2ij7roeDEm7laC9pT18eBZRW5fVDvz6 VsGM3LOorT8Qd8jzbVdkJluojUrRpnpseal/Jjk0cNWXWe+1VgFYkUR6exfv6e9HN5By MYW9OMsmYXrC2qJbjHMdMScXhQWtMoKnjuAY4xvOsZgwbfYE+RdN5HLogvo2o0gOWY83 fxgY6UylkJNAeHde/7gOQ5mRcM3QElv/hPVYqvjeoXWa7DaTXGfWFKkO1eQ0iNpVbMHp QJgA== X-Gm-Message-State: APjAAAXsk4190DjJ2e8DebDQqFa61WkqcxT/J3hhEPqLjhrrv4SuSn3p oI4K1CKgr3/EAPv1kImQCy0Tyj4nf3rt8jrPkcIulw== X-Google-Smtp-Source: APXvYqxvZ6DCuJBCeNxR3auFNov3NTAXDIXQSfdLVHpJMJbfWMIlCwjamuyGu/b6KnJ/AMQIG/TxBaSxbf/Noi5dYi4= X-Received: by 2002:a02:37c6:: with SMTP id r189mr13535732jar.118.1567121238698; Thu, 29 Aug 2019 16:27:18 -0700 (PDT) MIME-Version: 1.0 References: <201908291905.x7TJ5Bw8091371@gndrsh.dnsmgr.net> In-Reply-To: From: Ed Maste Date: Thu, 29 Aug 2019 19:27:01 -0400 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Warner Losh Cc: FreeBSD Hackers , fcp@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46KJfw1NcJz4JdM X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.50 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-5.54 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.993,0]; RCVD_IN_DNSWL_NONE(0.00)[50.166.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.55)[ip: (-7.04), ipnet: 209.85.128.0/17(-3.34), asn: 15169(-2.32), country: US(-0.05)]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 23:27:21 -0000 On Thu, 29 Aug 2019 at 17:32, Warner Losh wrote: > > In the past, if someone had any follow on work at all in their tree, the > reversion would be quite disruptive to that work. This sounds more like a problem with the tooling than an argument against reverting though. I agree that in the case where the fix is straightforward it's sensible, and in line with community norms, to just commit it. But in the case that a regression was introduced by a committed change, modern tools facilitate reverting and replaying changes without a lot of effort. > things quickly, imho, to hesitate a little on the revert. Especailly when > the broken thing is the playstation loader on powerpc that can stay broken > for the hour or six (or even days) it takes me to figure out why it > broke... Often things away from the beaten path don't get discovered for > days or weeks or months, and a reversion then can be extremely disruptive > if there's other changes layered on top of the offending commit.... Note that this isn't at all the issue under discussion in the FCP, which refers to issues that have already been detected by CI. For example, a commit which means amd64 panics on boot. Reverting quickly is a benefit in this sort of case found by CI precisely so that we don't end up with other changes on top that are difficult to unwind. From owner-freebsd-hackers@freebsd.org Thu Aug 29 23:35:44 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 14FDEE29B0 for ; Thu, 29 Aug 2019 23:35:44 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x82c.google.com (mail-qt1-x82c.google.com [IPv6:2607:f8b0:4864:20::82c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KJrb1pGNz4K6Z for ; Thu, 29 Aug 2019 23:35:42 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x82c.google.com with SMTP id b2so2233652qtq.5 for ; Thu, 29 Aug 2019 16:35:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=OuE5G/e6FXGMIkJBV8BP3Wk+5bkDJcfztwL1SvRTBzw=; b=wy+1S0BELR6E4b2udOu2XvcXrdWNJlpPtozUulVNWd7KlACMl7CaLkBeItIVxb68II J51mSc1MSKClJswNZUDjrXNWenBhTYRcAQsqWao8hU98dQFaQPeY2O9yActusBOKL0d+ CvtiLdJ2bN1MfX+UvjSxA+0SCoL3lvu016hphgRl7+Qu1H0hqkeVLWdEI8Mt3x7/O0/V dHYxR/ZGHfNjjsyaBDKp34+PWfs/WGTM1NZrAh6JT7CusNGkOlRBsh2AmJI1bIxz5OzZ pcq9LmIR5T9iAdOsT5ykP1QyIBk/qsyBjPwJmkeXod5F9FlKUBpXkssi66vwm9hwrbno onhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=OuE5G/e6FXGMIkJBV8BP3Wk+5bkDJcfztwL1SvRTBzw=; b=PDxZc4wBp+9LGMyyozK58oasRgu+WyKQLBVIaJoIZtdHyqE6sfyU2jguqIU6rATLzL yOfzb3AxeRieMWdsJYytXgFB+wyKHKFNWNB7m+IqktdhlPUy2P+c0flxNMFgtQs3UDy2 UFlynK/YLsOo4WH6EuonPgEPXPlIvurfsPSbmQQ2E0XJZtvF9G8UGXsCXHMhe8+NF6Ud P2WRC+THPQyXde7+0NSOvYHZw1puZmITJSUBFMg65UQsof3Cjzl5hejyiI0Xu4iaeBGF kkFxUqCxRgSX+8IOTtTNk2RbC7zkIzeMrXOWUbwFI8uWF3RPqeu8/w+pawYdaGW0Py+/ y3mA== X-Gm-Message-State: APjAAAXx7u2ASio7dK8ql3BVCEtUXd5q2PkF32feCzIVp8lk3IPyYNM/ 57z0b63vTP3OfbyrNRFatIGkOx6bJovtEgncJJ9/rg== X-Google-Smtp-Source: APXvYqzTMPTj69Ol2cCMDvT0Szma2ouUiUxKeSVLtqMCIutdNKm+dZxP09Xg8gGHjgbA8zLtCU++I6+o/7ECcIFRycQ= X-Received: by 2002:ac8:4602:: with SMTP id p2mr12594446qtn.291.1567121741918; Thu, 29 Aug 2019 16:35:41 -0700 (PDT) MIME-Version: 1.0 References: <201908291905.x7TJ5Bw8091371@gndrsh.dnsmgr.net> In-Reply-To: From: Warner Losh Date: Thu, 29 Aug 2019 17:35:29 -0600 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Ed Maste Cc: FreeBSD Hackers , fcp@freebsd.org X-Rspamd-Queue-Id: 46KJrb1pGNz4K6Z X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=wy+1S0BE; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::82c) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-5.89 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-0.996,0]; RCVD_IN_DNSWL_NONE(0.00)[c.2.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.90)[ip: (-9.28), ipnet: 2607:f8b0::/32(-2.84), asn: 15169(-2.32), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 23:35:44 -0000 On Thu, Aug 29, 2019, 5:27 PM Ed Maste wrote: > On Thu, 29 Aug 2019 at 17:32, Warner Losh wrote: > > > > In the past, if someone had any follow on work at all in their tree, the > > reversion would be quite disruptive to that work. > > This sounds more like a problem with the tooling than an argument > against reverting though. > We live in a subversion universe for the moment, so you have to view it through that lens. I agree that in the case where the fix is straightforward it's > sensible, and in line with community norms, to just commit it. But in > the case that a regression was introduced by a committed change, > modern tools facilitate reverting and replaying changes without a lot > of effort. > Sometimes yes, sometimes no. Even with git svn, there is a cost associated with it. The level of effort is not zero. Especially when one pushes several interrelated changes at once. If the first of these caused an issue on gcc, say, often the cost is too high to revert the whole chain. It's a lot easier to put in a fix and move on. > things quickly, imho, to hesitate a little on the revert. Especailly when > > the broken thing is the playstation loader on powerpc that can stay > broken > > for the hour or six (or even days) it takes me to figure out why it > > broke... Often things away from the beaten path don't get discovered for > > days or weeks or months, and a reversion then can be extremely disruptive > > if there's other changes layered on top of the offending commit.... > > Note that this isn't at all the issue under discussion in the FCP, > which refers to issues that have already been detected by CI. For > example, a commit which means amd64 panics on boot. Reverting quickly > is a benefit in this sort of case found by CI precisely so that we > don't end up with other changes on top that are difficult to unwind. > It's a fair example for why a simpleminded approach will create more friction than the current system. And there is a need for caution in expanding the logic beyond all but the most recent changes... Warner > From owner-freebsd-hackers@freebsd.org Thu Aug 29 23:36:17 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B497DE2A7F; Thu, 29 Aug 2019 23:36:17 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-io1-f44.google.com (mail-io1-f44.google.com [209.85.166.44]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KJsD1Rflz4KDy; Thu, 29 Aug 2019 23:36:15 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-io1-f44.google.com with SMTP id j5so10394996ioj.8; Thu, 29 Aug 2019 16:36:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Ua4zHBQZ8l/telU2EihwTZk/176grRwgom62ZiitBoM=; b=QgMYZy0lLoHhNqJmpdUCMCKpoT7UCMpohCHDKkIVvskY21DExp/HhBzOWsGjLS1Hnx vFgqnbLgE4i+Wtf54WibFxtKs1SVYzM8BIQPrZwTc1XhPmc7XQtf2vyM1amANllUmIBR UEhNF+M6irPcNJJEz1HVe7JwoMb0EKD3nmlOB57GpAARlR3w7GqvBzTUb7/mwC4Bk+Xv jM2YOjWPYn1MEZtw/nR3jvJ8q1bxei3VrUAMQiIdbPa6yX3krzzuPzaGKNtYxBfV6bhG jlBWOkIpL0jcE/aDClP1x8/s7sxNuQwqj+NLudzHWxCVeHRZIdtmTQfbbNDXl6rkPAST j9rQ== X-Gm-Message-State: APjAAAWO53v7g+734sk5SnxCy9gm0tnpt5FzYgefnE/vVFNxMzPOYjy9 qif2PlAla5qULtRAxPDZVsb+4HAxwPU7NnHX2hs= X-Google-Smtp-Source: APXvYqyIya+E6staTKdL0E9iswUIHwjX0ue+iYsK6/O1VQFGcLqT7fJ/JyYx9r+rdZYDTMGdj1rwFFymq2XTS/lYRfI= X-Received: by 2002:a5d:9b96:: with SMTP id r22mr1006520iom.17.1567121775027; Thu, 29 Aug 2019 16:36:15 -0700 (PDT) MIME-Version: 1.0 References: <20190829114057.GZ71821@kib.kiev.ua> In-Reply-To: <20190829114057.GZ71821@kib.kiev.ua> From: Ed Maste Date: Thu, 29 Aug 2019 19:35:57 -0400 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Konstantin Belousov Cc: Li-Wen Hsu , FreeBSD Hackers , fcp@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46KJsD1Rflz4KDy X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.44 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-5.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.991,0]; RCVD_IN_DNSWL_NONE(0.00)[44.166.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.01)[ip: (-4.31), ipnet: 209.85.128.0/17(-3.34), asn: 15169(-2.32), country: US(-0.05)]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; FREEMAIL_TO(0.00)[gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 23:36:17 -0000 On Thu, 29 Aug 2019 at 07:41, Konstantin Belousov wrote: > > More, I know that tests are of very low quality, which means that > brokeness of the tests is not an indicator of anything until root cause > is identified. "Low quality" needs clarification here. I can think of many attributes of a test that might lead someone to claim tests are low quality: - The test result is not consistent (e.g., a "flaky test") - The test does not actually test what it claims to - The test does as it claims, but there is no value in the result - Test coverage overall is insufficient (i.e., not an issue with a specific test) - The test has excessive requirements (run time, memory usage, etc.) - The test is difficult to maintain From owner-freebsd-hackers@freebsd.org Thu Aug 29 23:45:18 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BC82FE2DDD; Thu, 29 Aug 2019 23:45:18 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-io1-f54.google.com (mail-io1-f54.google.com [209.85.166.54]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KK3d4fxyz4Klf; Thu, 29 Aug 2019 23:45:17 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-io1-f54.google.com with SMTP id p12so10491877iog.5; Thu, 29 Aug 2019 16:45:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=0FSmYVskYbBYnGN185uLQL22o1T6D9MYbnRnoxDC67o=; b=YquEtH80LyW3R9CDO7QUZqCKeO5zjbWtVE6YWGMkcLMvlGXBfddG7Ce+fS1VC8IpHw JodIJyadRUhHtAFJijSvkHnoWd5sdUanOgKOQXgcLl0OuBq+nV/uj5DBF4W9xRBOkntF DwabNmKLapTfZQjcoTggiVhJ2VVICHst5+u8CLs7N+GzQb3Kt0TpqjYax8PVD9AzDQ6b yfej53kV8rH/C58O/YA8Nh+seaiLQl8wOAhuH6OPyyxsAyjSZyk3qAlxspwq0FHcNQgn kNjZxfwisDlQAnMgwFJXmPWpzYETFCScG1WizxG7TfU6vTeXH3XAYqno5FCkcJl+M9wa Dy6A== X-Gm-Message-State: APjAAAXgI3f8nh7PU5W2gVGtOtzkyf8JMfrfunKRYvRnxXbtheSaXOjs 7xZgqxAEJmS3X9ytFFnZGNIhcxBX3bhOxAx0exM= X-Google-Smtp-Source: APXvYqxgh0nMR4dvDtHiT+C0pqoSVClomevBvA1i9zsdwkPeAyVUjWvvFInhdvwOKiJlGFD0Rk5dDi2llkHn41/2ObY= X-Received: by 2002:a6b:3806:: with SMTP id f6mr1805827ioa.120.1567122316168; Thu, 29 Aug 2019 16:45:16 -0700 (PDT) MIME-Version: 1.0 References: <201908291905.x7TJ5Bw8091371@gndrsh.dnsmgr.net> In-Reply-To: From: Ed Maste Date: Thu, 29 Aug 2019 19:44:59 -0400 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Warner Losh Cc: FreeBSD Hackers , fcp@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 46KK3d4fxyz4Klf X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.54 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-5.49 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.993,0]; RCVD_IN_DNSWL_NONE(0.00)[54.166.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.50)[ip: (-6.78), ipnet: 209.85.128.0/17(-3.34), asn: 15169(-2.32), country: US(-0.05)]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 23:45:18 -0000 >> This sounds more like a problem with the tooling than an argument >> against reverting though. > > We live in a subversion universe for the moment, so you have to view it t= hrough that lens. Fair enough, right now the policy needs to accommodate the reality of the tools we're using today. Perhaps it's a failure of imagination on my part but I have trouble seeing how a revert would lead to losing work - could you give an example? > Sometimes yes, sometimes no. Even with git svn, there is a cost associate= d with it. The level of effort is not zero. Especially when one pushes sev= eral interrelated changes at once. If the first of these caused an issue on= gcc, say, often the cost is too high to revert the whole chain. It's a lot= easier to put in a fix and move on. The level of effort imposed on other users while the tree is broken is not zero, either. Certainly if it's possible to commit a fix and move forward that's the approach favoured by community norms. > It's a fair example for why a simpleminded approach will create more fric= tion than the current system. And there is a need for caution in expanding = the logic beyond all but the most recent changes... The point of the FCP is to facilitate the revert while the change is (among the) most recent, precisely so that additional changes don't build on it. From owner-freebsd-hackers@freebsd.org Thu Aug 29 23:54:49 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B248FE30D9; Thu, 29 Aug 2019 23:54:49 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-io1-f53.google.com (mail-io1-f53.google.com [209.85.166.53]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KKGd4Hbgz4LDV; Thu, 29 Aug 2019 23:54:49 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-io1-f53.google.com with SMTP id u185so6683695iod.10; Thu, 29 Aug 2019 16:54:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=PoIWWx4fSq3VskXld9iN/CzgUfJmJ9GAd0lWF1+bQzY=; b=kSxcHhOQY5xSZmi1v0CfPJolAfXwi3XG7HJUs94d4OLBJIxFd6AjE2Z00/gO+0bx1j 8WbuyskOQwWwmbZquBmvuHSCtAunVrStyFwGXwJYZAGqU6e/sPjoF7Y8huyOmkMAOcDe ob0ab0hCE/s4CrMDYmbcxlRWAqIDph35n7QbwUkGWjJwW1ndvYpvs7UFYLyg/lDlMRbl CF1buuiV/RUPCqiAdjy9zY9ean8t5n/3lH1jF0yUDXdDjmBMwCTYN9aac5p/rCyVWzaI CPU9LnO/DVIowH3Zm9spLAV2wOOyi7XazI1pkjyiRm5WJRuHzSSpyyPD5oyMeabZH/ni xZAA== X-Gm-Message-State: APjAAAX+VeJIuYqqYn5DEosT5L8shYZqK5pfz55x5QdcTDGNFSxVcZ2v DDkuNayOnJBXfJAVt51uBXj5GeGpt/ZODQ5l2uYcFA== X-Google-Smtp-Source: APXvYqxw0zz6Avaf3xljxYeS7M1ZnHfStnwytUNEyLTK25j9Swu5Ho1oyV3LW4zE0PPdjvTz/Ix8+G2XvsLN78LjbIo= X-Received: by 2002:a5d:9b96:: with SMTP id r22mr1084304iom.17.1567122886864; Thu, 29 Aug 2019 16:54:46 -0700 (PDT) MIME-Version: 1.0 References: <20190829114057.GZ71821@kib.kiev.ua> <412537DD-D98F-4B92-85F5-CB93CF33F281@FreeBSD.org> In-Reply-To: From: Ed Maste Date: Thu, 29 Aug 2019 19:54:29 -0400 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Marcelo Araujo Cc: Kristof Provost , Konstantin Belousov , FreeBSD Hackers , Li-Wen Hsu , fcp@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46KKGd4Hbgz4LDV X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.98)[-0.980,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 23:54:49 -0000 On Thu, 29 Aug 2019 at 10:47, Marcelo Araujo wrote: > > One last thing I'd like to say: We should stop to try to control FreeBSD > contributions, it is not doing good for the project. Let it be, let it go, > there is no owner, just go and fix things. I don't see things that way. Other projects I contribute to (like LLVM) have a comprehensive test suite, and a culture that expects tests to be run before commit. Many projects run the tests automatically as part of the contribution process (e.g. GitHub pull requests). Rather than being some bureaucratic hurdle this actually supports casual contributors - because the contributor has more confidence that their change isn't going to break the build or introduce a regression, and doesn't need to remain attentive for the next days or weeks in case of unexpected failures as a result of the change. From owner-freebsd-hackers@freebsd.org Fri Aug 30 00:00:59 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D809EE34CF; Fri, 30 Aug 2019 00:00:59 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-io1-f51.google.com (mail-io1-f51.google.com [209.85.166.51]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KKPk5QPVz4LV1; Fri, 30 Aug 2019 00:00:58 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-io1-f51.google.com with SMTP id n197so8538092iod.9; Thu, 29 Aug 2019 17:00:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=TZytS+qLdG33lUssuiKaFq2nPipy4oB5XPa0SwhYjtg=; b=MWyHBS0B8BapQAaXz7ql1Hnt0RBxjCrjIVBHlg3PRA0ZCV/c9lkafsWrsiqAXSkFGs 8AFGmC2+e2ErhHx8/BoZXJYLzjxi+7IA1EbOvA4IOTpyl2d2btdEoh4NZBPhHb28vyuq wScCp+W6K7lFnVr3/rXZvM4rzh1jyqbE3QAfP5L2queRInd6pbUFIP0vVSbg8tsGUx73 jLZg+DC+fDNNnBiArWHYdG9V03wtgeI3Wt6PTgfaiA9rXhHsmkFBcYheAX6UU9sTJETH rJIkjEFc0vnw754vk7qNNCqOsHyxbnwTxs3yMNihHzlOWVAgxvbvO3xRJKN6PMGHCJqY 8JhA== X-Gm-Message-State: APjAAAV9yLeeIUrjGHbxEtpvDg424LNpkBduMeawsB15VswURzG2FYCP nnLpnIkE7y7hkLVkoPC6lKqN2AdYr/4LJw2FwFs= X-Google-Smtp-Source: APXvYqwkSda+4/rES3/SE85Qy3NmavwPXy57WQVVKqnVZBKioRGX5foRbFpzBY2ZhvncpUKh5YMitnbu412iijsP9QQ= X-Received: by 2002:a5d:9b96:: with SMTP id r22mr1111435iom.17.1567123257701; Thu, 29 Aug 2019 17:00:57 -0700 (PDT) MIME-Version: 1.0 References: <20190829114057.GZ71821@kib.kiev.ua> <412537DD-D98F-4B92-85F5-CB93CF33F281@FreeBSD.org> <20190829144228.GA71821@kib.kiev.ua> <20190829153712.GB71821@kib.kiev.ua> In-Reply-To: <20190829153712.GB71821@kib.kiev.ua> From: Ed Maste Date: Thu, 29 Aug 2019 20:00:41 -0400 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Konstantin Belousov Cc: Kristof Provost , FreeBSD Hackers , Li-Wen Hsu , fcp@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46KKPk5QPVz4LV1 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.51 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-5.49 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.990,0]; RCVD_IN_DNSWL_NONE(0.00)[51.166.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.50)[ip: (-6.77), ipnet: 209.85.128.0/17(-3.34), asn: 15169(-2.32), country: US(-0.05)]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; FREEMAIL_TO(0.00)[gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 00:00:59 -0000 On Thu, 29 Aug 2019 at 11:37, Konstantin Belousov wrote: > > I fully agree with your attitude there, and understand your frustration. > IMO the right action would be to contact the committers who did the > relevant changes, first. Was it done ? What was their response ? If we had tests running consistently at the time an offending change was introduced we might know that :) This is exactly the reason I want us to have a large corpus of tests that are, as a rule, expected to consistently pass. As far as I can tell from a cursory investigation this is a flaw that predates the pf tests, and just happens to be demonstrated by them. From owner-freebsd-hackers@freebsd.org Fri Aug 30 00:05:24 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0EF80E3D82; Fri, 30 Aug 2019 00:05:24 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-io1-f54.google.com (mail-io1-f54.google.com [209.85.166.54]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KKVq23Pbz4Nyd; Fri, 30 Aug 2019 00:05:23 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-io1-f54.google.com with SMTP id t6so10519715ios.7; Thu, 29 Aug 2019 17:05:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=URxnojM7DYm+fSQKO6xIxHE6v96bixxiCb42ycf6FKo=; b=lUbR8XSKlXf2nIXW6/6wJaQU1Xl2x5sbTxMLHSjPzCe7Tpn0obN+y8VlB6UsrcVxpJ P2IwsOznnyxPnjeuu3Kxk5ylU6bg9swQriXISx3+mNufaGfqYUTCFgjiA3KvH+137aqj 2aY5dlkicc9fIy5pKBvMVPJyPoZ0zBRmzbbWILPnysim7jhittO8/S049ZEFaOfU8Drz XP/NLZnc41sOcG63I2hucqby6SMcn8tH4TFNGgmXCRTbSxdwvFbnGyL7t5gg9aHOtUv1 OIEK3DobKxfQoKzNyEPsLlGhMBsXHsn1QauoAH04tXXkd8TACGGtsI/Ld+fthwO8Yyqh 0FbQ== X-Gm-Message-State: APjAAAVJobdFZ90xNN/ndidBr8AvkZMrlNIrH/HnQ6JfoaIvXLqlr7wE 7Rq/x2bkN6Y44CEpTLgPjsDi7nUe0x+tBNaDri1buA== X-Google-Smtp-Source: APXvYqx0xvT5UR1XEZJhksJ4+mAwv4OS2uxkMCylQPLys5IW+ibcg9+8XtHj4VFYPV5wH9feBFo/B/RHzSEMMXXPzRU= X-Received: by 2002:a6b:ea16:: with SMTP id m22mr8361491ioc.115.1567123522546; Thu, 29 Aug 2019 17:05:22 -0700 (PDT) MIME-Version: 1.0 References: <201908291905.x7TJ5Bw8091371@gndrsh.dnsmgr.net> In-Reply-To: <201908291905.x7TJ5Bw8091371@gndrsh.dnsmgr.net> From: Ed Maste Date: Thu, 29 Aug 2019 20:05:06 -0400 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: "Rodney W. Grimes" Cc: FreeBSD Hackers , fcp@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46KKVq23Pbz4Nyd X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.54 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-5.46 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.993,0]; RCVD_IN_DNSWL_NONE(0.00)[54.166.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.46)[ip: (-6.60), ipnet: 209.85.128.0/17(-3.34), asn: 15169(-2.32), country: US(-0.05)]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 00:05:24 -0000 On Thu, 29 Aug 2019 at 15:05, Rodney W. Grimes wrote: > > Here in lies one of the fundemental problems, this view by some that > a "revert commit is something bad, it is kind of punishment". That is > not true. Reverts are GREAT things, they allow the tree to be returned > to a known state, usually quicly. The original commit is STILL IN SVN, > and a bad revert can guess what.. be reverted!. Let me echo Rod here. I'm also very happy that this statement was made by one of the original FreeBSD committers. Reverting a change is not an insult, not a punishment, not something bad - it's simply an acknowledgement that some aspect of the change didn't meet expectations. From owner-freebsd-hackers@freebsd.org Fri Aug 30 00:19:16 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9D844E45C8 for ; Fri, 30 Aug 2019 00:19:16 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x72b.google.com (mail-qk1-x72b.google.com [IPv6:2607:f8b0:4864:20::72b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KKpq2nRYz4Qd3 for ; Fri, 30 Aug 2019 00:19:15 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x72b.google.com with SMTP id f10so4674862qkg.7 for ; Thu, 29 Aug 2019 17:19:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=j+SRZ5kUCBdK4V6FTFHS6+HnVnHlwC8XmWCMrEVfwHI=; b=OWeFDN3bJkNaDnRPUwTvNCeugNyhD4xZ34GEAduGSdcAywKWcb7gIbmJcHopvj9qt6 W99K1qGIV7nmZirLgfqeUx0lPHmDAUVKMU6XdJy7/QShvY4+UBzM+kb4FhmOegIm7f86 YJwwpgxNmRGQFhPhrnHyB909IMqi3BolNl2TaLLhj9HC/jCzOw/pN2DKVNOkrUiVh2mJ 90/QDLeTJKN2vXdidemzUg6fjyzrj9G/DZN860AeAMNooPxPmbHdyghjf62qhfNcMupe CTnl+dlI6H6sOfE7OWf75PuJLzvf9f665Od+20nkrtwXTv1W0k5sOQFUCwcQC7G3h3rs 0MMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=j+SRZ5kUCBdK4V6FTFHS6+HnVnHlwC8XmWCMrEVfwHI=; b=JUVpYw/6GtkecPHGeURv2do9NbBX1XV8fD0RkPBL6gWRJ+aciM0iFBZyw1TYuCJJgv O6Jd+NBIO3/T01KVvTrTirBGEtyKTroLoeZHYMqZLc7RE7qwB4EoYomxpsww/eJoXwR0 fR5DMqn37fpkEFv75em+zMnLKkXfyM/+gmZmh9+azgtuoQ4HnsfHdl+3a17gXNWMLOkw bD9CQXj3KHmT9hf5JajBGsVR6Ht+F2T3hzfqHBEzMcIUcbh3Tmjpi6dSbfLZKh13Se4D BnFzdrl7j/psynMXmcfAfEXxjdfD1TLBsCw9+BLuUosOtRnmlESDc7uo2qkJcybJx2yJ xNJQ== X-Gm-Message-State: APjAAAWnp9Q31WCt2Pgu0+jPD+mM0lXV49VwZx8rtiydcgN/enXhQ1+F df8Q+rvJ2l+Gz+nackS490uB/X97IHMxskRHUP+C67ImZuM= X-Google-Smtp-Source: APXvYqyp1HtSIHpIEncgQbk5/qWyYzY5hebGyJRUdq3pgDtVRBbNWjTHqbXWgSN1y2Nh0mhffhqtpWVAY+ot7BFBNn4= X-Received: by 2002:a37:8902:: with SMTP id l2mr12348810qkd.380.1567124354153; Thu, 29 Aug 2019 17:19:14 -0700 (PDT) MIME-Version: 1.0 References: <201908291905.x7TJ5Bw8091371@gndrsh.dnsmgr.net> In-Reply-To: From: Warner Losh Date: Thu, 29 Aug 2019 18:19:02 -0600 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Ed Maste Cc: FreeBSD Hackers , fcp@freebsd.org X-Rspamd-Queue-Id: 46KKpq2nRYz4Qd3 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=OWeFDN3b; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::72b) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-5.93 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-0.996,0]; RCVD_IN_DNSWL_NONE(0.00)[b.2.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.94)[ip: (-9.47), ipnet: 2607:f8b0::/32(-2.84), asn: 15169(-2.32), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 00:19:16 -0000 On Thu, Aug 29, 2019 at 5:45 PM Ed Maste wrote: > >> This sounds more like a problem with the tooling than an argument > >> against reverting though. > > > > We live in a subversion universe for the moment, so you have to view it > through that lens. > > Fair enough, right now the policy needs to accommodate the reality of > the tools we're using today. > > Perhaps it's a failure of imagination on my part but I have trouble > seeing how a revert would lead to losing work - could you give an > example? > I've twice lost work due to a hasty reversion in the past. In my case, when I did the svn update it ended badly due to the conflicts that were generated. Most of the times I've hit conflicts in the past, it's just been the work of a conflict. When the work was larger, svn got a bit confused and I wound up losing several chunks of work. I was able to redo it, but it was annoying. It's why a super-fast automatic revert without contact with the person that committed it is a bad idea. Of course, if that person falls down in fixing it, I support doing something. Tools have improved, and perhaps this is a case of once bitten twice shy... > > Sometimes yes, sometimes no. Even with git svn, there is a cost > associated with it. The level of effort is not zero. Especially when one > pushes several interrelated changes at once. If the first of these caused > an issue on gcc, say, often the cost is too high to revert the whole chain. > It's a lot easier to put in a fix and move on. > > The level of effort imposed on other users while the tree is broken is > not zero, either. Certainly if it's possible to commit a fix and move > forward that's the approach favoured by community norms. > I agree. My pushback here is against the notion there's zero cost to a revert. Of course we need balance the damage to others vs the impact on the contributor. When the impact is in -current on a fringe platform, we need to not over-react to fixing that by back out. > > It's a fair example for why a simpleminded approach will create more > friction than the current system. And there is a need for caution in > expanding the logic beyond all but the most recent changes... > > The point of the FCP is to facilitate the revert while the change is > (among the) most recent, precisely so that additional changes don't > build on it. > Agreed. I just don't want to swing too far to the automatic end of things, and to apply some level of judgement when there's more than one change involved. Warner From owner-freebsd-hackers@freebsd.org Fri Aug 30 01:35:50 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0E25DE5D91; Fri, 30 Aug 2019 01:35:50 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-no.shaw.ca (smtp-out-no.shaw.ca [64.59.134.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KMW846DWz4VHg; Fri, 30 Aug 2019 01:35:48 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.67.125.17]) by shaw.ca with ESMTPA id 3VpHi2s32sAGk3VpIitl9V; Thu, 29 Aug 2019 19:35:46 -0600 X-Authority-Analysis: v=2.3 cv=WeVylHpX c=1 sm=1 tr=0 a=VFtTW3WuZNDh6VkGe7fA3g==:117 a=VFtTW3WuZNDh6VkGe7fA3g==:17 a=8nJEP1OIZ-IA:10 a=FmdZ9Uzk2mMA:10 a=6I5d2MoRAAAA:8 a=YxBL1-UpAAAA:8 a=duxrOwH0z9cosxiZrWsA:9 a=wPNLvfGTeEIA:10 a=IjZwj45LgO3ly-622nXo:22 a=Ia-lj3WSrqcvXOmTRaiG:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTPS id 98DC4725; Thu, 29 Aug 2019 18:35:42 -0700 (PDT) Received: from slippy.cwsent.com (localhost [127.0.0.1]) by slippy.cwsent.com (8.15.2/8.15.2) with ESMTP id x7U1Zg6M067244; Thu, 29 Aug 2019 18:35:42 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Received: from slippy (cy@localhost) by slippy.cwsent.com (8.15.2/8.15.2/Submit) with ESMTP id x7U1Zfh7067241; Thu, 29 Aug 2019 18:35:41 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <201908300135.x7U1Zfh7067241@slippy.cwsent.com> X-Authentication-Warning: slippy.cwsent.com: cy owned process doing -bs X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.7.1 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: "Kristof Provost" cc: araujo@freebsd.org, Konstantin Belousov , FreeBSD Hackers , Li-Wen Hsu , Ian Lepore , fcp@freebsd.org Subject: Re: FCP 20190401-ci_policy: CI policy In-reply-to: References: <20190829114057.GZ71821@kib.kiev.ua> <28934eb780342605090bf365ac3a2e0d522256f5.camel@freebsd.org> Comments: In-reply-to "Kristof Provost" message dated "Thu, 29 Aug 2019 17:09:32 +0200." Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Date: Thu, 29 Aug 2019 18:35:41 -0700 X-CMAE-Envelope: MS4wfCbf9EUat0hnmBN/VxddWxCz2VbiMs9RNbe9T2Rpl2FPyFF9yIjEFS3WPs2NbFDiNbEojxO8rUhOpBG8drXgSquMESIPMpGSN5Ofhq2zAo9nYNFmh6Le K6V3XJS434syjkd5snUoXy90o3NGoEfEgGlEtUyDhAZ570GTu82U0t1EwyEp69EDXkM1J0zD6+qaP+772Sxu1KI4OFfyZ+7+A1iQLp4ipWecHEHmuZsJ4B5X MWPI73lCwlqC3H7dKfzeMUa/BFZsW1/QrMlvOXr0RUdlCDGq+ZYkLglYuhoQPVrtPn2HHCgxd25MdU6MHnvfTw== X-Rspamd-Queue-Id: 46KMW846DWz4VHg X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of cy.schubert@cschubert.com has no SPF policy when checking 64.59.134.9) smtp.mailfrom=cy.schubert@cschubert.com X-Spamd-Result: default: False [-4.95 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; HAS_REPLYTO(0.00)[Cy.Schubert@cschubert.com]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; REPLYTO_EQ_FROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.95)[-0.952,0]; RCVD_IN_DNSWL_NONE(0.00)[9.134.59.64.list.dnswl.org : 127.0.5.0]; RCPT_COUNT_SEVEN(0.00)[7]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6327, ipnet:64.59.128.0/20, country:CA]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-2.40)[ip: (-6.35), ipnet: 64.59.128.0/20(-3.13), asn: 6327(-2.43), country: CA(-0.09)]; RECEIVED_SPAMHAUS_PBL(0.00)[17.125.67.70.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 01:35:50 -0000 In message , "Kristof Provost " writes: > On 29 Aug 2019, at 17:01, Marcelo Araujo wrote: > > Em qui, 29 de ago de 2019 às 22:54, Kristof Provost > > escreveu: > > > >> On 29 Aug 2019, at 16:42, Ian Lepore wrote: > >>> (And I don't think breaking a test counts as > >>> breaking the build.) > >>> > >> I fundamentally disagree on this point. A test failure is, just like > >> a > >> compiler warning, a precious gift that should not be ignored. > >> The more distance (both in terms of time, and in terms of the people > >> involved) there is between a bug being introduced and it being > >> detected > >> the harder it is to fix it. Test accelerate detection of bugs. If we > >> do > >> not take test failures seriously (i.e. as an indication something is > >> wrong and should be fixed) the tests will inevitable become useless > >> in > >> one of two ways: we’ll either disable failing tests (which is what > >> we > >> tend to do now) reducing test coverage or we’ll have a test suite > >> with > >> many failures in it, which makes it useless as well. (As with > >> compiler > >> warnings, the best way to keep them under control is to consider them > >> to > >> be fatal errors.) > >> > > > > Could you elaborate where is the "fundamentally" you disagree? Where > > is the > > fundament? You guys are introducing something new, yes everybody knows > > about test, it is year 2019, but nobody can come with new rules tha in > > hours we gonna revert if you "dare to don't fix it". Sorry, this is > > not how > > people test software and fix it. > > > I do think that breaking a test breaks the build. Something used to work > and now it doesn’t. That’s breakage, even if it’s not as total as > it not compiling any more. I agree. A broken test is an indication of a regression. If a test is not a regression, then it is not a valid test. Having said that, 48 hours is a little draconian. Maybe 48 hours if no one steps up to the plate but if it's actively being worked on with end in sight I'd give the person working on it a little space. What about weekends? I notice commits tend to slow down during weekends. I suspect people who work on FreeBSD at $COMPANY for a living treat it as a job, which it is. However this should be considered. OTOH (arguing against myself to point out another consideration), it is foolhardy to commit a new feature just before a weekend and go camping (or be unreachable for whatever reason). A committer should be reachable N hours after a significant commit. Maybe 48 hours is justifiable then. (At $JOB we don't allow significant changes prior to vacation or even a flex day unless there is someone to cover for the person having done the work.) My point is, we pick a number like 48. What's the rationale? What is the desired result? Can the desired result be accomplished without a 48 hour auto-revert rule? Should it be 48 hours during the week and 72 over weekends? -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few. From owner-freebsd-hackers@freebsd.org Fri Aug 30 01:58:17 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3E650E6294; Fri, 30 Aug 2019 01:58:17 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "troutmask", Issuer "troutmask" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KN141ZZhz4Vy7; Fri, 30 Aug 2019 01:58:15 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) by troutmask.apl.washington.edu (8.15.2/8.15.2) with ESMTPS id x7U1w6nP016918 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 29 Aug 2019 18:58:06 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.15.2/8.15.2/Submit) id x7U1w6q2016917; Thu, 29 Aug 2019 18:58:06 -0700 (PDT) (envelope-from sgk) Date: Thu, 29 Aug 2019 18:58:05 -0700 From: Steve Kargl To: Cy Schubert Cc: Kristof Provost , Ian Lepore , FreeBSD Hackers , araujo@freebsd.org, fcp@freebsd.org, Konstantin Belousov , Li-Wen Hsu Subject: Re: FCP 20190401-ci_policy: CI policy Message-ID: <20190830015805.GA16894@troutmask.apl.washington.edu> Reply-To: sgk@troutmask.apl.washington.edu References: <20190829114057.GZ71821@kib.kiev.ua> <28934eb780342605090bf365ac3a2e0d522256f5.camel@freebsd.org> <201908300135.x7U1Zfh7067241@slippy.cwsent.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201908300135.x7U1Zfh7067241@slippy.cwsent.com> User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: 46KN141ZZhz4Vy7 X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of sgk@troutmask.apl.washington.edu has no SPF policy when checking 128.95.76.21) smtp.mailfrom=sgk@troutmask.apl.washington.edu X-Spamd-Result: default: False [1.18 / 15.00]; ARC_NA(0.00)[]; HAS_REPLYTO(0.00)[sgk@troutmask.apl.washington.edu]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.45)[0.445,0]; IP_SCORE(-0.14)[ip: (0.06), ipnet: 128.95.0.0/16(-0.07), asn: 73(-0.62), country: US(-0.05)]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[washington.edu]; REPLYTO_ADDR_EQ_FROM(0.00)[]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.62)[0.623,0]; NEURAL_HAM_LONG(-0.65)[-0.647,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_SEVEN(0.00)[8]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:73, ipnet:128.95.0.0/16, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 01:58:17 -0000 On Thu, Aug 29, 2019 at 06:35:41PM -0700, Cy Schubert wrote: > > What about weekends? I notice commits tend to slow down during weekends. I > suspect people who work on FreeBSD at $COMPANY for a living treat it as a > job, which it is. However this should be considered. > A committer, who works for a $COMPANY and has no intention of looking at the mailing lists or CI reports until Monday, should not commit on Firday. What ever happen to common sense? -- Steve From owner-freebsd-hackers@freebsd.org Fri Aug 30 02:01:12 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7E4A3E63F9; Fri, 30 Aug 2019 02:01:12 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-no.shaw.ca (smtp-out-no.shaw.ca [64.59.134.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KN4Q2qRvz4WFn; Fri, 30 Aug 2019 02:01:09 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.67.125.17]) by shaw.ca with ESMTPA id 3WDqi33ZosAGk3WDritoMk; Thu, 29 Aug 2019 20:01:08 -0600 X-Authority-Analysis: v=2.3 cv=WeVylHpX c=1 sm=1 tr=0 a=VFtTW3WuZNDh6VkGe7fA3g==:117 a=VFtTW3WuZNDh6VkGe7fA3g==:17 a=kj9zAlcOel0A:10 a=FmdZ9Uzk2mMA:10 a=7Qk2ozbKAAAA:8 a=iKhvJSA4AAAA:8 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=GD4Lj2ni4_LfeXk_s_gA:9 a=CjuIK1q_8ugA:10 a=HNhVPpsFFhwA:10 a=1lyxoWkJIXJV6VJUPhuM:22 a=odh9cflL3HIXMm4fY7Wr:22 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTPS id EE3C3754; Thu, 29 Aug 2019 19:01:04 -0700 (PDT) Received: from slippy.cwsent.com (localhost [127.0.0.1]) by slippy.cwsent.com (8.15.2/8.15.2) with ESMTP id x7U214i0086083; Thu, 29 Aug 2019 19:01:04 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Received: from slippy (cy@localhost) by slippy.cwsent.com (8.15.2/8.15.2/Submit) with ESMTP id x7U214qn086080; Thu, 29 Aug 2019 19:01:04 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <201908300201.x7U214qn086080@slippy.cwsent.com> X-Authentication-Warning: slippy.cwsent.com: cy owned process doing -bs X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.7.1 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: Warner Losh cc: "Rodney W. Grimes" , Ian Lepore , FreeBSD Hackers , Marcelo Araujo , fcp@freebsd.org, Konstantin Belousov , Li-Wen Hsu , Kristof Provost Subject: Re: FCP 20190401-ci_policy: CI policy In-reply-to: References: <201908291905.x7TJ5Bw8091371@gndrsh.dnsmgr.net> Comments: In-reply-to Warner Losh message dated "Thu, 29 Aug 2019 15:32:17 -0600." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 29 Aug 2019 19:01:04 -0700 X-CMAE-Envelope: MS4wfCI6Bkf8Ut4g4MzXQp1XYy6ido4ceg6QBsesfQUY+aMQgaACX9ydSCxK/0xBNpj1JgJNpgdrqFAlnwpWR7bWWfmMt6xAWe1lYac7jxAcBPVcRzjMjeSL 5y8ItgPdDptRmOG4F6GNUBNBbe+9YykOS4laGW8S7DggEwYagGaoeFS998kXhOCYHH+6OcJnbRS+LPaZLzF1Hd/N/qnAKkIBtGsJraoGjVKz2PhQ1nJJSCNG ncuXy6NRannaA/DQivkgak67BVWVKBnrCu1RuV1nzvMpz/XmIzVsJni6ja5x4dnyfoVuX2uPewpprvjKG5jGOVHQep68RHe9FPzLYZzezDFQC4hiOVOM7r9f yW7AxOPYOweVgafdCJap9it7RXBYgYEP/GvV2qOhpaV0EPhcIOk= X-Rspamd-Queue-Id: 46KN4Q2qRvz4WFn X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of cy.schubert@cschubert.com has no SPF policy when checking 64.59.134.13) smtp.mailfrom=cy.schubert@cschubert.com X-Spamd-Result: default: False [-5.03 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; HAS_REPLYTO(0.00)[Cy.Schubert@cschubert.com]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; REPLYTO_EQ_FROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.95)[-0.952,0]; RCVD_IN_DNSWL_NONE(0.00)[13.134.59.64.list.dnswl.org : 127.0.5.0]; RCPT_COUNT_SEVEN(0.00)[9]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6327, ipnet:64.59.128.0/20, country:CA]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-2.48)[ip: (-6.76), ipnet: 64.59.128.0/20(-3.13), asn: 6327(-2.43), country: CA(-0.09)]; RECEIVED_SPAMHAUS_PBL(0.00)[17.125.67.70.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 02:01:12 -0000 In message , Warner Losh writes: > On Thu, Aug 29, 2019 at 3:26 PM Warner Losh wrote: > > > > > > > On Thu, Aug 29, 2019 at 1:05 PM Rodney W. Grimes < > > freebsd-rwg@gndrsh.dnsmgr.net> wrote: > > > >> (unneeded context removed) > >> > >> > > In either scenario we end up reducing test coverage, which means we?re > >> > > going to push more bugs towards users. > >> > > > >> > > > I totally agree. This is an overly-bureaucratic solution in search > >> of > >> > > > a problem. > >> > > > > >> > > > If this needs to be addressed at all (and I'm not sure it does), > >> then > >> > > > another sentence or two in bullet item 10 in section 18.1 [*] of the > >> > > > committer's guide should be enough. And even then it needn't be > >> > > > overly-formal and should just mention that if a commit does break > >> the > >> > > > build the committer is expected to be responsive to that problem and > >> > > > the commit might get reverted if they're unresponsive. I don't > >> think > >> > > > we need schedules. > >> > > > > >> > > I do feel that?s a better argument. We?ve always had a policy of > >> > > reverting on request (AIUI), so this is more or less trying to be a > >> > > strong restatement of that, more than a fundamental shift in policy. > >> > > > >> > > >> > We don't have a policy to revert commit, actually revert commit is > >> > something bad, it is kind of punishment, I have been there, nobody > >> wants to > >> > be there. Stop to push this non-sense argument. > >> > >> Here in lies one of the fundemental problems, this view by some that > >> a "revert commit is something bad, it is kind of punishment". That is > >> not true. Reverts are GREAT things, they allow the tree to be returned > >> to a known state, usually quicly. The original commit is STILL IN SVN, > >> and a bad revert can guess what.. be reverted!. > >> > >> IMHO the project as a whole needs to overcome its fear of reverts and > >> start to use them for the great and powerful things that they are. > >> > >> This connection of bad and punishment needs to stop, and the sooner > >> the better. > >> > > > In the past, if someone had any follow on work at all in their tree, the > reversion would be quite disruptive to that work. Most of the time it's a > lot easier for me, with a lot less friction, to just fix issues that come > up after the commit than to revert and prepare a new commit. Sure, it's > possible, but it can destroy work in extreme cases. *THAT* is why I'm > firmly in the camp of giving the original committer a shot at fixing things > because it's much less disruptive to them, and generally we can get a fix > into the tree faster. It reduces friction and encourages people to fix > things quickly, imho, to hesitate a little on the revert. Especailly when > the broken thing is the playstation loader on powerpc that can stay broken > for the hour or six (or even days) it takes me to figure out why it > broke... Often things away from the beaten path don't get discovered for > days or weeks or months, and a reversion then can be extremely disruptive > if there's other changes layered on top of the offending commit.... > > So the whole reversion issue is a lot more complicated than 'oh, it's still > in svn'. There are real high costs associated with being too quick or > liberal on the revert and those must be weighed against the damage the bad > commit is doing.. I think that's why we need to define the problem first. The justification of the arbitrary numbers of minutes/hours isn't clear. I see there are possibly two trains of thought here which need to be separated. 1. A general frustration by some. 2. A tool, a solution to a problem, I am unsure if it is related to #1. Why do I see this as such? The problem statement beings by saying that FreeBSD has a CI system that performs compiles and runs automated tests. In the next paragraph it says sometimes changes break compilation... This tells me that A) we have a solution which we discuss in B (the problem). To my thinking we need to approach this from: A) we have a problem, maybe backing it up with some stats some evidence of sorts. Then explore one or preferably two solutions. Not to be hard on anyone and keeping my emotions out of it, they way the problem statement is structured reads to me as a solution looking for a problem. That's not to say we don't have a problem (nor am I saying we do have a problem either). The problem statement is structured to a foregone conclusion. I'd structure this by stating the problem (paragraph 2 and the bullet points, though I think the problem needs to be explored a little more), then discuss some of the timing issues regarding the fixing of the three types of problems (compile, panic, and regressions, of which tests identify regressions). I'm not saying that there is or isn't a problem but the problem statement as written doesn't convince me there is a problem. It's leading me to a conclusion. -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few. From owner-freebsd-hackers@freebsd.org Fri Aug 30 02:02:31 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F1076E6658; Fri, 30 Aug 2019 02:02:31 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-no.shaw.ca (smtp-out-no.shaw.ca [64.59.134.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KN5y54r1z4WSX; Fri, 30 Aug 2019 02:02:30 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.67.125.17]) by shaw.ca with ESMTPA id 3WF9i34D3sAGk3WFBitobe; Thu, 29 Aug 2019 20:02:29 -0600 X-Authority-Analysis: v=2.3 cv=WeVylHpX c=1 sm=1 tr=0 a=VFtTW3WuZNDh6VkGe7fA3g==:117 a=VFtTW3WuZNDh6VkGe7fA3g==:17 a=kj9zAlcOel0A:10 a=FmdZ9Uzk2mMA:10 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=kjVYAFG_BOeK_iIailIA:9 a=CjuIK1q_8ugA:10 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTPS id 88E38761; Thu, 29 Aug 2019 19:02:27 -0700 (PDT) Received: from slippy.cwsent.com (localhost [127.0.0.1]) by slippy.cwsent.com (8.15.2/8.15.2) with ESMTP id x7U22RlP087313; Thu, 29 Aug 2019 19:02:27 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Received: from slippy (cy@localhost) by slippy.cwsent.com (8.15.2/8.15.2/Submit) with ESMTP id x7U22RaV087308; Thu, 29 Aug 2019 19:02:27 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <201908300202.x7U22RaV087308@slippy.cwsent.com> X-Authentication-Warning: slippy.cwsent.com: cy owned process doing -bs X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.7.1 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: sgk@troutmask.apl.washington.edu cc: Cy Schubert , Kristof Provost , Ian Lepore , FreeBSD Hackers , araujo@freebsd.org, fcp@freebsd.org, Konstantin Belousov , Li-Wen Hsu Subject: Re: FCP 20190401-ci_policy: CI policy In-reply-to: <20190830015805.GA16894@troutmask.apl.washington.edu> References: <20190829114057.GZ71821@kib.kiev.ua> <28934eb780342605090bf365ac3a2e0d522256f5.camel@freebsd.org> <201908300135.x7U1Zfh7067241@slippy.cwsent.com> <20190830015805.GA16894@troutmask.apl.washington.edu> Comments: In-reply-to Steve Kargl message dated "Thu, 29 Aug 2019 18:58:05 -0700." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 29 Aug 2019 19:02:27 -0700 X-CMAE-Envelope: MS4wfGKDwjGnXx+/Juz+HJ9qJ3XckpyZciCBnS1eUmv9SKKHNiBizNQMDwzoNALcWHDxEiNiGaZx65ZpdluDq3U0HsW2kqRXBLsAV+eHWHhZ2VCp5EU/IBje NI9bzyxBh9wiCCS/vvMwbJLAH0CloE/H55WxphH2eixWcY2MGkf9weutAhnvbNlA0RhtW6r5N0KxUhc4apdPcg+GN5d/jKbOkV6UfhNmgc/JLP/XFGK/MoxN 3TKIUjeRP/GwQA5fAh9y+lHVGqyT1eLZ8yGvcmgCmo7EfsVsnXbetDHKtfFCLWvPUthMLGi79hkS1SfkmKiIEqWNK/2oe/8sy5E+mIScxTrysQTqaFzFrGFB 77FrHttO/oWPpxTUtJCd/HpWNuG53Q== X-Rspamd-Queue-Id: 46KN5y54r1z4WSX X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of cy.schubert@cschubert.com has no SPF policy when checking 64.59.134.12) smtp.mailfrom=cy.schubert@cschubert.com X-Spamd-Result: default: False [-4.99 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; HAS_REPLYTO(0.00)[Cy.Schubert@cschubert.com]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; REPLYTO_EQ_FROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.95)[-0.951,0]; RCVD_IN_DNSWL_NONE(0.00)[12.134.59.64.list.dnswl.org : 127.0.5.0]; RCPT_COUNT_SEVEN(0.00)[9]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6327, ipnet:64.59.128.0/20, country:CA]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-2.44)[ip: (-6.55), ipnet: 64.59.128.0/20(-3.13), asn: 6327(-2.43), country: CA(-0.09)]; RECEIVED_SPAMHAUS_PBL(0.00)[17.125.67.70.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 02:02:32 -0000 In message <20190830015805.GA16894@troutmask.apl.washington.edu>, Steve Kargl w rites: > On Thu, Aug 29, 2019 at 06:35:41PM -0700, Cy Schubert wrote: > > > > What about weekends? I notice commits tend to slow down during weekends. I > > suspect people who work on FreeBSD at $COMPANY for a living treat it as a > > job, which it is. However this should be considered. > > > > A committer, who works for a $COMPANY and has no intention of > looking at the mailing lists or CI reports until Monday, should > not commit on Firday. What ever happen to common sense? I've worked with and do work with people like that. -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few. From owner-freebsd-hackers@freebsd.org Fri Aug 30 06:55:48 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3ECFDC4342; Fri, 30 Aug 2019 06:55:48 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46KVcM5ZFpz3G0q; Fri, 30 Aug 2019 06:55:47 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x7U6tYGl041508 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 30 Aug 2019 09:55:37 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x7U6tYGl041508 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x7U6tY3t041507; Fri, 30 Aug 2019 09:55:34 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 30 Aug 2019 09:55:34 +0300 From: Konstantin Belousov To: Ed Maste Cc: Li-Wen Hsu , FreeBSD Hackers , fcp@freebsd.org Subject: Re: FCP 20190401-ci_policy: CI policy Message-ID: <20190830065534.GC71821@kib.kiev.ua> References: <20190829114057.GZ71821@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-Rspamd-Queue-Id: 46KVcM5ZFpz3G0q X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.90 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; NEURAL_HAM_SHORT(-0.91)[-0.906,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 06:55:48 -0000 On Thu, Aug 29, 2019 at 07:35:57PM -0400, Ed Maste wrote: > On Thu, 29 Aug 2019 at 07:41, Konstantin Belousov wrote: > > > > More, I know that tests are of very low quality, which means that > > brokeness of the tests is not an indicator of anything until root cause > > is identified. > > "Low quality" needs clarification here. I can think of many attributes > of a test that might lead someone to claim tests are low quality: > > - The test result is not consistent (e.g., a "flaky test") > - The test does not actually test what it claims to This. In fact, for code base like OS which provides an API/ABI to the large corpus of user code, it does not matter what exactly test claims to test. We are interested in any behaviour change. But very unfortunate and deeply engraved wart in many tests I saw is that they depend on unspecified or non-guaranteed API details to claim success. So detected behaviour changes often appears to be irrelevant. When I was (forced to) look into test failures, it was 50 vs. 50 % of test bugs vs. some legitimately catched issues. > - The test does as it claims, but there is no value in the result > - Test coverage overall is insufficient (i.e., not an issue with a > specific test) > - The test has excessive requirements (run time, memory usage, etc.) > - The test is difficult to maintain This is too. My main complain is that to debug a test case, I must strip all atf* to be able to examine it under a debugger. From owner-freebsd-hackers@freebsd.org Fri Aug 30 07:25:05 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1EEFAC50BC; Fri, 30 Aug 2019 07:25:05 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46KWG8602Sz3HPK; Fri, 30 Aug 2019 07:25:04 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x7U7Ou0a047434 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 30 Aug 2019 10:24:59 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x7U7Ou0a047434 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x7U7OtPu047433; Fri, 30 Aug 2019 10:24:55 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 30 Aug 2019 10:24:55 +0300 From: Konstantin Belousov To: Ed Maste Cc: Warner Losh , FreeBSD Hackers , fcp@freebsd.org Subject: Re: FCP 20190401-ci_policy: CI policy Message-ID: <20190830072455.GD71821@kib.kiev.ua> References: <201908291905.x7TJ5Bw8091371@gndrsh.dnsmgr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-Rspamd-Queue-Id: 46KWG8602Sz3HPK X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.92 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.92)[-0.920,0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 07:25:05 -0000 On Thu, Aug 29, 2019 at 07:44:59PM -0400, Ed Maste wrote: > >> This sounds more like a problem with the tooling than an argument > >> against reverting though. > > > > We live in a subversion universe for the moment, so you have to view it through that lens. > > Fair enough, right now the policy needs to accommodate the reality of > the tools we're using today. > > Perhaps it's a failure of imagination on my part but I have trouble > seeing how a revert would lead to losing work - could you give an > example? If you have a committed change that is the precursor for later series, and this series is not yet committed. Then rebase or merge of the master with the base change reverted causes much trouble. > > > Sometimes yes, sometimes no. Even with git svn, there is a cost associated with it. The level of effort is not zero. Especially when one pushes several interrelated changes at once. If the first of these caused an issue on gcc, say, often the cost is too high to revert the whole chain. It's a lot easier to put in a fix and move on. > > The level of effort imposed on other users while the tree is broken is > not zero, either. Certainly if it's possible to commit a fix and move > forward that's the approach favoured by community norms. > > > It's a fair example for why a simpleminded approach will create more friction than the current system. And there is a need for caution in expanding the logic beyond all but the most recent changes... > > The point of the FCP is to facilitate the revert while the change is > (among the) most recent, precisely so that additional changes don't > build on it. Also, I have to admit publically, that my changes were blamed by CI so many times, when they had nothing to do with the breakage. I forwarded such mails to ci@ regularly, then stopped doing that. Also, I believe that imposing requirements on committers to not offend CI is not fair if committers cannot _easily_ verify that the change does not. Give us a way to e.g. mail pgp-signed message to ci-check@ and get the reply 'ok/fail' in reasonable time, or a button on phab to get the same response. Even if I bother to recreate CI env locally (I do not), races are different on the CI machines. From owner-freebsd-hackers@freebsd.org Fri Aug 30 07:56:48 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 083B4C5EEE for ; Fri, 30 Aug 2019 07:56:48 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KWyk664Qz3Kp1; Fri, 30 Aug 2019 07:56:46 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id x7U7uZ68018158 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 30 Aug 2019 07:56:36 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: brde@optusnet.com.au Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id x7U7uUWI054482 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 30 Aug 2019 14:56:30 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: FreeBSD Security Advisory FreeBSD-SA-19:23.midi To: Bruce Evans References: <20190820201257.7A9D41F8B7@freefall.freebsd.org> <20190830133855.W1405@besplex.bde.org> Cc: John Baldwin , Freebsd hackers list From: Eugene Grosbein Message-ID: <5961a5af-d36c-4b8f-20c1-e7054b0149f4@grosbein.net> Date: Fri, 30 Aug 2019 14:56:21 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20190830133855.W1405@besplex.bde.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 46KWyk664Qz3Kp1 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-4.64 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; NEURAL_HAM_SHORT(-0.96)[-0.958,0]; IP_SCORE(-1.58)[ip: (-4.13), ipnet: 2a01:4f8::/29(-1.97), asn: 24940(-1.80), country: DE(-0.01)]; FREEMAIL_TO(0.00)[optusnet.com.au]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 07:56:48 -0000 30.08.2019 11:03, Bruce Evans wrote: > The patch preserves some historical mistakes and adds some excessive > verboseness about probe failures. I'm still waiting for jhb to reply to > mails on 30 Oct 2015 and 23 Jan 2018 asking for a review of this patch > or better a complete fix. Hmm... Maybe additional notification worth it :-) From owner-freebsd-hackers@freebsd.org Fri Aug 30 10:54:55 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8E6F0CAD85; Fri, 30 Aug 2019 10:54:55 +0000 (UTC) (envelope-from shreyankfbsd@gmail.com) Received: from mail-yw1-xc29.google.com (mail-yw1-xc29.google.com [IPv6:2607:f8b0:4864:20::c29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KbwG4W1tz402s; Fri, 30 Aug 2019 10:54:54 +0000 (UTC) (envelope-from shreyankfbsd@gmail.com) Received: by mail-yw1-xc29.google.com with SMTP id 129so1600117ywb.8; Fri, 30 Aug 2019 03:54:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=rG84iHW20UnQWjFBxnIGlbit+94wIaS18Dg1CC5iWQQ=; b=qxPeUuZKVdBA5FQ7N5T5c0Q2miMlDe9UVNsgdJex/bdOHHVWyDkr53iPvClK3YqaOV 5UD40Pgqyf9Y2ZJePudAfTH5sh6aH8ejetFvftPXwBOVpQ4Ew9p8hEk7bkgziiIAkEnF UB4Q/m81WP791kD+lZ81LPit7ceUpLrduXiXwUbaueWx9wx9zRuQ0IpEn0WMY+1bfqi9 KQA6bX5U+FDiSkd1JQbrVCEJuIcxbZFxDrK4vM+Q9PYf06qLGBWSyrGZ6JQepK87o1CI JdMlyThv4Op8ix/h0i419oadQUnyhGLqCxIVzPLztGjjbJYl1Y2/8K8HjyZqXRMTsJhM i4PA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=rG84iHW20UnQWjFBxnIGlbit+94wIaS18Dg1CC5iWQQ=; b=J5DhyMTgNIYasVShbCGxkXqoQp5f1kwYbeD5EmWyWS1WPX6uBJPV1ZFKiAKLWXHcFX UETITjs5DOD0RQbnQp9ZajPOSDkSgOB6xqUHpTotCtSWpnqI/53rGUa7wnRH/DFSLi0T iUsz/p5mtkPuSjPbgWYhjKoSS9obxWhtlMvH8OvBGwjTSLcqocDeJizJTqmsae0ZhK4l D3CnxqwhYGfVTzLHlK82QTuv0+ASojwOfNHIZpI9zOs8Tf+QTx1yxQQELnGrQvs59F0q jD9YDF4CF++0EozDPOJQVt4CBcoUg7R8VYWxsOudQKeVws4yybZpwm8Rf8Ysk7530nMf LKyA== X-Gm-Message-State: APjAAAWuFkqatVI/SgwBJDvXHZXAYP9e5HgTdppS+XuLs4KaFq+FSznE WC0ZHr+ddtuIn6YsfAWTOkI0OkaM0by8EtQul4+/cO4= X-Google-Smtp-Source: APXvYqx5MXMH3I2pRxWpzcg7EuAv504Ks+UjYFiKHY63yX/VO9W53nRjX3v1zpv51QPUy2yh9f1ojYpQJOZOLNblTsI= X-Received: by 2002:a81:4d43:: with SMTP id a64mr10535729ywb.131.1567162493160; Fri, 30 Aug 2019 03:54:53 -0700 (PDT) MIME-Version: 1.0 From: shreyank amartya Date: Fri, 30 Aug 2019 16:24:41 +0530 Message-ID: Subject: hwpmc ktrdump: failed to resolve ktr symbols To: freebsd-drivers@freebsd.org Cc: freebsd-hackers@freebsd.org X-Rspamd-Queue-Id: 46KbwG4W1tz402s X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=qxPeUuZK; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of shreyankfbsd@gmail.com designates 2607:f8b0:4864:20::c29 as permitted sender) smtp.mailfrom=shreyankfbsd@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; IP_SCORE(0.00)[ip: (-9.39), ipnet: 2607:f8b0::/32(-2.84), asn: 15169(-2.32), country: US(-0.05)]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[9.2.c.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-1.00)[-0.996,0]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 10:54:55 -0000 Hi, I'm trying to collect traces from hwpmc module. I compiled the module with -DHWPMC_DEBUG option and set the sysctl kern.hwpmc.debugflags but I'm unable to retrive the traces using ktrdump. I get this error message: ktrdump: failed to resolve ktr symbols I'm guessing that this error occurs because the symbols for hwpmc module (compiled and loaded separately) are not a part of kernel. If this is the case, then how do I get those symbols. In the ktr man page its also mentioned that ddb and gdb can retrieve ktraces. I found the ddb command show ktr. But how do I invoke ddb (without a panic)? I'm unable to find how can I use gdb to view the traces (I launched kgdb but could not find a command to show kernel traces). Any idea on how to get ktrdump working? Or get the kernel traces. Thanks Shreyank From owner-freebsd-hackers@freebsd.org Fri Aug 30 12:28:44 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 983A1CD924 for ; Fri, 30 Aug 2019 12:28:44 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46Kf0X4Fwcz44pG for ; Fri, 30 Aug 2019 12:28:44 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp.infracaninophile.co.uk [81.2.117.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: matthew/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 0E4615D0D for ; Fri, 30 Aug 2019 12:28:44 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from liminal.local (unknown [IPv6:2001:8b0:151:1:3835:e07e:a9b4:98f6]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: m.seaman@infracaninophile.co.uk) by smtp.infracaninophile.co.uk (Postfix) with ESMTPSA id 398638D25 for ; Fri, 30 Aug 2019 12:28:42 +0000 (UTC) Authentication-Results: smtp.infracaninophile.co.uk; dmarc=none (p=none dis=none) header.from=FreeBSD.org Authentication-Results: smtp.infracaninophile.co.uk/398638D25; dkim=none; dkim-atps=neutral Subject: Re: FCP 20190401-ci_policy: CI policy To: freebsd-hackers@freebsd.org References: <201908291905.x7TJ5Bw8091371@gndrsh.dnsmgr.net> From: Matthew Seaman Openpgp: preference=signencrypt Autocrypt: addr=matthew@FreeBSD.org; prefer-encrypt=mutual; keydata= mQINBFJIL80BEADi7/VbnnErDU6pjEhI/SzEZ/HbDRkJ5g7HroAtqIRm6nj8ZwOAgZ/2ZnWn 5F+fXTuLsG0FLNtkd17FoVcuCi5e/GPliXI5cmamV7E1Yz4T8UsJ7RQolimyxVexccKd16Tc AA7B9bFlJSKkBUSD0buj7VjT07xWhRzu6Vgi5r0UjLALYJz977uZA0F1aOGOXREDEAOhdcNc kSNjynqAwDA6dCT1Elpi4key1fYjv4jyDF+GU/YXul2Y/rguA8FCkHd9vyym5eAsLQ5mG00V V9fkEHIpH5KorNVnl/ufHXnkZqmHAZVpFDcrshb7aZ/pL45PXyWgLj+e6etelgj3a2bZi0JF cVdXCnBZVP2oIyYblM11ugTbfCwodORU8a5KfPeztMdAtDr4e+32NTrPdPi5rLT+GUsYz+PL 3A3m3u8bdsFp40DlIrBtSByVjqERxcfhphrEB4J8BXHUG7OAtXkZMlW/PGKDwXJq0O6Z5Tcg YHAoEiSWbXiexHgXNJyP+sqnIlhLWhSJGeJ+C83wqI6oYlZUCW00NkPxcIHnQPV/z+5wQVci TMyaWC2YCIHz4Ljs+TnwWMz0E8PNFDfHVbQ0W4PRGV7gRAqxfL+yKufauIEGbEq8rNDbSwL3 bcUCxR4ZDlaUEUwT4J8naf7rjdgiEYHs2Ig3jeK1+ER4FPG1sQARAQABtDBNYXR0aGV3IFNl YW1hbiA8bS5zZWFtYW5AaW5mcmFjYW5pbm9waGlsZS5jby51az6JAlcEEwEKAEECGwMFCwkI BwMFFQoJCAsFFgIDAQACHgECF4ACGQEWIQRyz6whebywJLW1RZADb2ye5/OevwUCWttU4QUJ DFmAlAAKCRADb2ye5/Oevwb5EACipbOazgwl5IbqkQI4gELpCh5dqDASS9DQqAD35n/cI91P 0lrYcdyCQbOXadQi5bswnP4AcJqX83mITXbcApDdxVxHujw7VODI069eV3/I9Qz72mHYYAAj w0CHNx4bKED2YCSVS6+jV5hq2sywNEUxL+4I218Oc+IsLts62m4tQ8UxX9fQ2H1kQOvdrYpj x7je5qJX/yujLc+9WWZ8ZBSdP/HVJUEdRgQotwAlgfMp3mRQEE73MAJisG/olj/dSxd+oHIP NbJt1yxMqhZekuEGqZpm3tWvqYgpGcEXdhphJSxeK6oLpTLghuAb7/WdOBrpfL7c2OQYBgOw DK+7Io9NBt/d/rCxL39jmUONW8ohrhnNQ2SALnyYTvZgruxA4tXxOOyM9up0/8mB5E8YC9ML 5YuxRPNTXYeWCexa0zktnkCgT7PhS33evf5gsA0B9Snv7TFCFN9adPAdHlsppZIWfTHDG8e2 Jik8PmvsUG34XNif5k6Ui3++2ZA8ZoKvOyLeomuno1hN8yk1APw8SbX1SPNz9UVbl8W/YgGj 3GhYOuQt4HcMiLyTby6R4lC4nsBaHS1MX+57f6Zxzf2wNjSKxiJK9qS7azbu/GxpafNhbz1Z +iUDIaJkRWA1Gs8C7SMcfVsI5zDtvqHGYtTCgooVMYJ6vRyB68M4bljUYMxRTrkCDQRSUUGj ARAAsPHwcnupWuOqYbboiYwZnd6dNRSUzMxIXN8vkdkrDfw7DvV9WYuAC9IGJ310N0otfh9A zGDiCPRbKl0YayJ2BIgsFzyAavA/kCCRLP5hMZ1mKkZ4K8Fs16EvtmarzPibSBfDQ0wcwzNf nSL2gZVG1JwRHHZ9TtiUsuAIh0R/qRh9+8AcFkS5Pfxb1PzJC/YuWOdlj6cO58u+2FfmNiGm oB6kl1LahmbtGgO8GRInkOYUYlWSUAA4Flw4FzWHBkEGv/STAp++KAZu2Tdl5UZH9iXm+Hsf 4sqt+/ILJketmO2RK2o2ECVwE2a/hQdOjjqmcscd1M5znweKSCk6dR/K4Cv05bZ7KVRCm2vK vuEBpltm/43/ls7OnFwz1UVswX9ch9t5tgSwbGxtTWJ/Mr3ybCz0EE4WaJBI8HTuVZWaJwXM ozz26BZCOV56flkZjDuyRhvRjZG+QhdbbumBDpa6wu3MCjSG8wn4RlNjuQdjDCo6bdqyovGg f8RW6UNCmStZkpTZYZfs8MTEcltmaFiJQjnY39pWa+Fp0aWwcwOVlAkp2wX6FzQeIEbPW515 vAlCjXneJIN7jss4Y2QJtFFQaCw0c+NloESFFhCLvYBhMPf2kccnDu25VRupkLp6njQs94Nf jtSb8mzOa2EhAHY81pRfdetOPosi23P6zIGKLXkAEQEAAYkEuwQYAQoAJgIbAhYhBHLPrCF5 vLAktbVFkANvbJ7n856/BQJa21VJBQkMUG8mAonBvSAEGQEKAGYFAlJRQaNfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDY1M0E2OEI5 MTNBNEU2Q0YzRTFFMTMyNkJCMjNBRjUxOEUxQTQwMTMACgkQuyOvUY4aQBNlUBAAlCLRtOug Y70Q3lkGsFSNJZm9oqPJGorOsH+emDdsiZSe5Ut5P2MG+XlIofQOfxvupltzw2pFuJOvHEMS 0rod6lLJ6joInhf0ZQH3P6jF/d2Y8iR9+2nqBtUf27OsHVLRMd/5WHVgyMjjyNBq0urIdv4E wV8Y9CDtGBGeiYyMstaBxHdEH+oM9VZB92lv485p4V8t8k1BgNn7UjQzOMBlITAB7WsUcXGi zTjMMe1tX/IT+f00I4PWAn3w5q8ldvtsWf+muVpIaGpZBMrxBEPxYBD3WGMxiymthQQxgZAB 03GatfLjzixld5Zn8WuGiPOxOTBkJAudhxPvfkO+3jgLGSa7TN46HgNH36OdeEr4SMdspR0i 0lmW1hwHmpmyw3XYLy4BwmhuV9z1XQN3qab8FBxOpxcCxnbO4HoDgXAahQbRNSA7umzz+I7S UcZVnCCG3hCG4BLxklZhBw4RmUtRHiL8vu+MPKrcBnbZ8uJ2s3E6mhB0yM0UnA3pYhAysgwB q3n9jLYN0atzVmHL8Fxjyc7z1EJPgqFdfHfMYl/eLYmCuGNfMsSGlH9O7tWoE10qkDlLmNB7 jbiJNgTf9rc50QKKUqumqp4a1UMEnt+7yf//JqUD7Jf0iJrglLgUyPKSY5te9rJqHPy1wIXT 6pChY5ic8jmtXKsCZaaxL8rEsq0JEANvbJ7n856/RNkQAKiZK5wNuRyNJS21MUJxnP7biEW4 1QuGhV/7Ryw5XXIor8H7SZHCnVR1fCYnJWRwRYn0SyZGoERW/57rgibf8/gkPw741AkCKOhL TDNgvNriEjfWj3I0X6M90AZXhcnGVJTS/moV65g4lUo6jX1GiJyTCD4b9SLyNDzPgiWO2I3W R+Xf/W81PK1820CN7HpIZUrLfGF+Nr6kXUxeOeSpi7ZMB/p3e7ZSzY0Lp7PFqGfL9N1Jg26X 8DVaf/Em0AorutLx84DqqMfO02ySaCq0B83VYzbNB3Ascy4c2JNIvwMiyUbsOEzDKkqB3sYb 0iJtnty9DKvMaLps00eM1+GcYpLsspY4NZQeJTVC+WetRqzFM4k2JH1q3hwymYgIsxDam6kn U3m0bN19WLQYmS5HLPZbkmtpm3P49g6KLFxZHzklS7x8VUOMJ3O97xXScBC9bePB3tqQRDSs wX3YmIywTYVInEeFleNaXH3UoS3Dhw7KP3i/BNreWDM+oZhbc2OkgWzQzXfT+l17EcP9/xML 0CIgM/cJPwMOrKrdqgfL6zAYDUK0IGFgRoxgnAbnpPHCr7ykrELNLbGtnzchzCxnbIyrSVAb m+Dm5MnjQRiNFXbuvpkuVVFqo6a0OhX1cwTuCIzSEfSggRaOOEqXTk559dDOXDqVx9lVKniK vbGzkmhAuQINBFJRQiABEADC0axEKC09VCYGgsH20lUwtAXd6VUVCNENBlW+MXQYsKfCLqO+ XP6vM0pA+sSswaBeSB/Eu3XgdKhuYGKHqAOo4wyKvwk3h9IWmgVNMM8ZQFi/PP2ya56/tuWZ 7kkG2M2OfWQpnBHa97wSN0KWDjZHrQXQMggDq5EqimNc2+hFaB2zIGrP0tjXVrHLJEmJRLq2 ugTxpGKLlNOtBNEsWmiN+MafXpKM6HLDq1scCvrhRICheBsnGtcyGaErwpjNaLA70I0+B552 DfTj+PICOGCMnp4jlP6rmVG7RifZoE5DrkcdTim/IU0pLaO/Epts5lwDodEOW9CKQFH8dswT bp6xhKJf+y1dIwhoOIkEUspoME3rgLtn72+QQW4jw/4pjA7MQu9VOF9bUN/nxTfyn/Rct3Bq sBZPJURdorewPgoBsPxMaA7t8JRoRyuVwXGMacw+wdmv2lldsdUOGokSCB596FoXAcKWndiY dgNjMWJaODy2va9Vlv65hGQRXWcoI2ytMCSwSzslly+V+0jo0ZWoUpd+6BuYRvG1QUW5/Fco aPPJsr/UfU0jzg6bCAw/xw1nuGaiZTqNiNjklrGIKyi0UyY28DGGADn3j9obY7pOrI9nFicc NtxURyhmgHP9tiTYNTVaGPyJh+WV3ZH/Yb7TStZadLoWb5vXAs0DQj+qnQARAQABiQI8BBgB CgAmAhsMFiEEcs+sIXm8sCS1tUWQA29snufznr8FAlrbVUkFCQxQbqkACgkQA29snufznr9M zBAAvn4C8wWYyiObQbqgaAm8GjqlSi0lGEv7ydmcu2ElAAyD0dnxbEMKEGgBpQumGD8/1pdZ FYw3EIKWiazpvMVw+6fFz9GZdviuM1refUYm3duDejaNoH75zmIG9LRTOJ6RBkPd3oQznT40 X5K+ARqLaJDPAzjb6DH7HYINlvNvf89M4CVN0gofv7dcCqtBTF8CtXB3iG0cFAis/12PwpfH 3YzWq529jnJJCLChTD5eEBi2JNLzQRHMeqy8D4Bnkb+Ahkwgbzs5GXGYaXoZeyFKThTAK/sg eJ9Cz15azfKW+EWMUOcvCurqz2QajlLe04N9mU4vPp92VTo274CtfIg/shSguYXnEZ0I/sz3 VFn3Kn2bRYeRu6PyusNUsQ397Uw5wDVmqzQqz+MnOkP6xAJjOvnD05cdj17G4rJ8gTgmzDSA 6v0AfzhUygy6Qf0UgrWrFaFIL4zQWsp9sap/QTMm92SBhLOE/Kc7nkkueEeVp0TtbkWByxLq 77Gbp0m4iZB8zylaac118hY+/vJ87aTuKF4CiCcezaI5FMg8/VVczO7/LV/n8Uu8QUOYEatR cfOB2JNXxpI/LqXVzvXpUidJbwpXY2aZprgzGhahBocuRL9jY8qp4in5CkhyU+rZyHkpQMHI +i45KRHO5GDSDMQcDF2LYGRbDUMg7G1MYTJwzsG5Ag0EUkgvzQEQANi5h27KsPhVw6AKlUo8 htPapW7b4RS26/z2pJe1IJ+lejrD5LveuRxdO3V+5hxqdBMEYNuQRmOlgsjiXkM5XFIgBeEF VGBaDv5yKPZXNfqIJC2nNehcR+rWHq84yrVb/MAvEvfQTvn3GeCTDd51xYnZYVO0An44TLLe 9cKL/i5d4I7flz/NK4DMpSqBRs0z7Tj9uF22LtYDJhNnQPolF4f+ADRLGMsbNHpCKwLcuzCR NlWN+eTY9peGZEfDoJT39u4wdg2ut9aSTv3B+l5HHkfYSS2gNf5yQ3YOVbQp/D6vZvNBCS0n Y5G5ApFil2ZAdoqfllqeQ74eH/dEPqOK1LCiBznKPHoLvTAJgA9v+Lhb9qw1jbIVD56Y88ZW c2iONscDlN2dboAYXGu3pcc8KNFkfc/j3MKRfq6N2l+t/n4ueebtLZypDJ3v9X7cQAkaW90R DhEuPpvvd+MEZGDYH3ZtIokqXZ3G3yiAy4M4TGXg4jX2pQ8ccXciimcp3DaXvqcV/SKnF20Q l6lm0r9sNp8ZBWUkLeMnDnpMdSjlONGuG9TsM50gaDi+kJuy9/fnlA0UGMpQNmBc1wsNAHl/ Q3ObZHUQtsZZN0gYEusDHpNC87SHodMS5YTc/eKx02asEoIoue/vUejkI6dvHWZv93+13y3c ZBhHyfF6SEr5dNkjABEBAAGJAjwEGAEKACYCGwwWIQRyz6whebywJLW1RZADb2ye5/OevwUC WttVSQUJDFmA/AAKCRADb2ye5/Oev9SOD/48JvgAf/PkjW0+TTE5vDaqdlEmNBu3K/vFX4T7 u0YT+qzLGUGYUvISiti9Dl7dV8kTg/Yr20EbHpj2a1Iys03YbR3mn/p6dv9abyqkaSESHN/g PPk1rlEi/j3lyoQsjDN6bpBEwT7Kbgri+Lwtkwp0vGm8I5AOguGlnCuNqsJ2jnHJ6YnEaKKp imIkr8wJVWxmx0OfnZxWrhMr5txD2DG675r1/IyOkU6SnApoD15+fJQmrsSmCKo3cZUMvM5Q 9lUJgdKuC89jJ1NujCzk7SC/EP6xSW0KFGzpqK0leIfh1riQ8DNs9CWreLANKtq35qbDUeGy BHwki0krsRRuNfg+0c+Rc5XOl+vuGmwfblKguIkAKSMSsjslXHqom+9s+mhOqJUSjAHsazlL BkVn00DfooDQBeeOwDlRwmQi+xcV3FomZMf5+4ARmsfzGtRIiJp5pfjek/P9vjeW+UqlE2az teXCmaK0G2LaLVVNnJzrUVQAqpA5eMtd3Ay8IGlhbrfznmAplgUH0aYhR1twIbUF8MeyQYIH fofR+lOnp3/vufJFZWve4S6tbK/OA69+Xr4wKAG95XBw03qZtPFbWu9yk5AYuS02U4akBhFv NfSx4Bs2rcrXZh63VBrlNqecueJdOQiQuY6nGoUa5fiE9glZF5ib9PVa522bBwaI2mW1tbkC DQRSUUKTARAAt6FH3HbDFoumOWUuJlDgOQs3wdp2n3IKv7gqzbDdgaoWW7hDTvjO0Cb6p2PG UKEoxMQQoIdDO0pQ9rgr4Sh4VSVC9WMO/fUwqdrIs2nACIg4OwvNhIccW08S+N72f+yuXWOQ /dv79cwruE26/BEXgIP09MYcOWwcUCXzOoUR3er+jzcsN9uFjcsBVUJLIEru1askHRzCUa5P 9S9GAFBwN49HC5IJWEzdLP27FjjOG5UG3+QZahHrjG1i6S3bIYXtaGsqNyfkp9Is7Wpj2kk+ s9Ua+YMG/V5YVlbANIexa1yr75p1W9biqXpCWnB3TaHSfI0G1t9w8K2qhR/Z1/YLIcRzZ2aH JnvbzJYw5Cs1jfNpFytbASsxj0rbReouftlBvVWFRxsZ+oG1ZXL64/SVKMZAnfBNxd1uajp+ HtoQtYoTu88la6zcdnAhOD5JdOntN2VF8iQnDfPgkidfuSZ1C059xaRPTSRJBgMRDtOlDxgz 7Pxx/7L2jwxRY1dq6NGioflY7CCpGc7bi1K6xnf3lBL8X2nGpRAVsg9Lx1ShIWkgNbTAcPXp XcXlJ1xqz8HS8Twadh6gIfk/RNchBIED9lkVCKHYp/XQb8T8vMwn/kTWUm5WlPkQUFQN4D1b 6+dJw4bwn/wiRS8did1MU1OytJB6tljfEUCx0uKkzqr+33MAEQEAAYkEuwQYAQoAJgIbAhYh BHLPrCF5vLAktbVFkANvbJ7n856/BQJa21VJBQkMUG42AonBvSAEGQEKAGYFAlJRQpNfFIAA AAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDE5 RjE1NEVDQkYxMTJFNTA1NDRFM0YzMDAwNTEzRjEwRTBBOUU0RTcACgkQAFE/EOCp5OdNFg// ZqeVdGoKkMvALPzZjGz84+6l0kcMxSN4TfWmec0YpSmDEzCw4/SZoGqHlZb8lcTevmNrNXg6 c+wVw6P+Ycl20Nzb98Kt9C5sz+zGVmPPK+3O9gaPnEqlIKnnbxKXXNHQdd8Mf0UTpifMqX0I kWOqhe/tQKGoQ9+feKvLIaToIe/NjosW6vJ9YAgFqZ0015zwbElhMNFmgDMOI2SgjBZ9ngP1 U82Mqb7/7G9GxHtnwuJBSnPJgN8tav2O9uWPC0N8deyZBH4y9ERBPTFMc46wjkW030olcq7g 4hZ55rpPIEyGQZCq4u1gGibbiQJZEyUQT7BJm70/PeUr3uNjPlQODV/lF5TBvqGHEmlSQfo6 Yb/QQx07CK9bvhUSO2XP3ybS8JwoMZlgZzZcjiPiQF9ot6152/Cp/XrsKgtk+fg5ARZpyywR lQk1JCHRZvhgXIxqNYA04uwdPFcLI4vPiDaLS8mhXHLRZsSpHmIBqqrnam5Lq7iDc39UZrSJ MM40oy3iAOI2B7AOCbzxRuEplJd3E/tEqrnFGcPVN+h52ka74lEyfkwA2RrASWJJcXLN3/Vs izEj8okepefzjU/UPnU8sirzeWWo8Z4uKddovk//NwAPUJbee4vZLjYE6MWdpEoZP9CZXbtI PWuc9Djg16aHOgv44JPokDMaHA27A4rw2KwJEANvbJ7n856/SPkP/1bGUde7lnRTNd8c0ZrU tEi+OOibKyh7BjLUpzlihj3rGl9ljAF0eCdBrL1We3MDDcyi+XO7VZLiecZTlG6LLXFvEFjY pyPRx3bXlWk1/ahEiBoLWxedseNdFrO+H5XX6ODmKFFLhXgpsXnAxtM6Mxmrx0CGW4qzfUi7 Vsqj86gqlcet0/k5RqPMAhrGX5fNnQNWSAwumeFKM8UgDpKY0u7M2tS07B0ozXOSpqGTSJhX 6Ld2Nl95CL3wbSGuh1pDUOysAnzK5Rl/OQ9LtYpWomAKg6yn7gKYij5XmekAg/E+ybr5Gyx2 PgMQUGtuNmBRWP1qKtVUbrOekiuNz7kpdrP7M2O7i/cxWjGpVtjDNWuGkFgY3c+sKKawBma8 1K4rg044nkGwFX98vfEHVGu+HOd3D+Mv47nv4LQvzynBG/YflwaPmLhpw7HCPvpa4W7y8+5A KxDqWlM2NvrLwmwbmz9dQMGtjnNRm4uHfPX8AyzBoMtDrxNLIvDYlLqh+G2Q1shNNNdRNXn9 Z1pvri6KAHmH9GlISuM/jQfItout+Gtx9QUlNX3aIsdScTLA3jnMOpHcALCGI+XMiBNaVuYU xHgHh+MNYhmjQZZqASBCvVj1HyibDPZa/iQ4DBGBRlJb+8saPPqYVDQhosWSF20aJKwepZII OFjpMgmCIqZAnqK4uQINBFJRQrgBEADUWFag56O3CaycayGght1rYWYz7P9/3s7OlqAuEAId 8/kSz8jXzAb/Qb6t0247a2MD0gxnjgZQy2OiQOsOTrc31L6tUrLVATL5Q3oKIh9hOlNMA+cR jsgY3UmMaSw+Gftp64EJDBQwBXWT7CSUEJw4PqzwMPiTHRkmqQfzdfNagFJVqZ0e+cznoLzI 9WvkccwLW1kicBYEysX5yOXUQ9/PcKqRWcbxLFznJ16JsxL1DeUct5WRWUxECY2rM0t+AkNR a3NpzskiMUSzFhiGmJo9yyy1RS4drjMhEn/IcM1sO21ZF/WWuUVkul65qngFnaFDDRQ5lU3A agWhLhmppmK/yabSVfqz38B1APoBWuldYprslTbAOJrL2xFtiH7m9VYbP2aGdwr9V/C27kiN Wnm/lYzP9Z+dTFkxw2V+BOjiLWzDDD6pEE7YDhiPyoopadOyXtoJf3aK1OI+DBu3piBA/CDD DvavruM+3mjxUxcOo8w8rMaJzDUDLG0yOyhKWef3UW5ly3CKXe8+m/MZe0GavNBJt0ObLQpP mnn9b2kP/xS0ssszo8uzlfSMiGi9AedAoRQ7vFXfI0MBb0M8gJ6Ht/+j1b5Al9ABeeA3PRuu +aBJwBRdFp4AV5BsCa0Qb3aqVJUPuBvtY56aWWB9sSfQ1qeu/loRxkJbHhaPJswscQARAQAB iQI8BBgBCgAmAhsMFiEEcs+sIXm8sCS1tUWQA29snufznr8FAlrbVUkFCQxQbhEACgkQA29s nufznr+YBw//TJtAC9d/FYQQHKQg/QOEkcAL8Qx4HA2SICnhKqv64jPcYIUYocOO8Qayh+IV Da6MGkbsWdweUFuexMsW+17dqETfQjUApx32TUwF44WgIEfARLW2zRdRcXfsT4A2sQJCvNJr JnH3lywiJi+V848Q4sC3sSJREpcJd07oc2jxSKZyYZ1DBPfK1MyiwcBt2uFCTXdyFMham2aY LDP2JYvFP08tjTUAIKhe4B0bPTtldCf5sH5q8xrpaHnKHf0n7qMmK7NtGW/9R6WiCruiNsLn O95fms1tzKKfA4QXIYCEWl8XsRKwp51HZDjQu/KxPsjm6BL4eThnae9t3Zs5J0LiPxoFbN+p W7anft3YCeezB8+gus7I1Rn5yJMRyYRRVHtZZTBDQfoDqHgLY14GYtFGOT0IR/OuAzYM1CoM vVExgqVWixDwF5RH1OHO1TANqTGcrRm1lvasCWIphpoQVtkN4/PXGa+NhzsRmr/c5OUYxQNr oE8cdsK8mOIBRz9D2JpF7d2nr1X+vA4zk2JL61aCnc62BfSYNZWhCcOPJZUhFT9BqAkew0kk JzQ3jwHGAhfcfozTHoFsD08qAW0OUriEtH+EOXl+dYbjlNUjFPjJu49cZbtp/1TpsYOBdME1 QLM1TPanYXa7tb+IrRZN+Oi9i9VVym16DK7q21k3j0qRC0s= Message-ID: Date: Fri, 30 Aug 2019 13:28:40 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="QxkYVnuAhcV7vTIA1FrtnORyYqBLqkbFV" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 12:28:44 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --QxkYVnuAhcV7vTIA1FrtnORyYqBLqkbFV Content-Type: multipart/mixed; boundary="Y7bfoMCM9RxDhmqyQNihR9nDXm8bK0RLA"; protected-headers="v1" From: Matthew Seaman To: freebsd-hackers@freebsd.org Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy References: <201908291905.x7TJ5Bw8091371@gndrsh.dnsmgr.net> In-Reply-To: --Y7bfoMCM9RxDhmqyQNihR9nDXm8bK0RLA Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: quoted-printable On 30/08/2019 00:35, Warner Losh wrote: > On Thu, Aug 29, 2019, 5:27 PM Ed Maste wrote: >=20 >> On Thu, 29 Aug 2019 at 17:32, Warner Losh wrote: >>> In the past, if someone had any follow on work at all in their tree, = the >>> reversion would be quite disruptive to that work. >> This sounds more like a problem with the tooling than an argument >> against reverting though. >> > We live in a subversion universe for the moment, so you have to view it= > through that lens. Hmmm... given our declared intention to switch to git, how relevant will this FCP be given the sort of workflows typical with git? As in a typical scenario would be that you work on a topic branch, and one pre-condition for merging your branch into the mainstream is that it passes the CI testing? Also, implicitly in that scenario, that a standard part of doing any piece of work is to develop tests covering any relevant changes introduced. Cheers, Matthew --Y7bfoMCM9RxDhmqyQNihR9nDXm8bK0RLA-- --QxkYVnuAhcV7vTIA1FrtnORyYqBLqkbFV Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEGfFU7L8RLlBUTj8wAFE/EOCp5OcFAl1pFnhfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDE5 RjE1NEVDQkYxMTJFNTA1NDRFM0YzMDAwNTEzRjEwRTBBOUU0RTcACgkQAFE/EOCp 5OeinA/8CI/nYOCI599RbU50WyysGwIdSAo79xsHcZA/CWPnIyxO4AuysA+wvvwI f2ovfzpADagKc0pis06kkIz3AilV8iPlI4ORXiHlBIdjyvaV3eZ6SwwsFOZKXGAT QYtU0CwqOT35OAhS1V8vGy9jhr4WhL+i+CZpMGt+hTmuquzkad541xlcOx+bsFVr 2IJKwVPb7nWQRjrE+tJXbaDndtgK5p4XeidsfG0dW44mfqE7H2vqAE3HWoDj69zU N0E41VoELVgk5ZYq6XLSHAlS6MRStFFrVcREdtcDOinVMuFRlNLon0/K7SO8VT21 P/YlnJM2U/2WuhErEeP3FrnIg7CW6O2JSk92bydI7GJbZDvRA1FN1PKCZ/ShjUXm gg3Wvw/aVIcPuBoMEE8fUUG1NKtfp6EVos8gG+sbrSAxOPGOo6qdM6a5gjPRkpE+ PktyT3hkoCJZV0C+mwN6Px55cDSdCwoL7sDjThYTHJuo2f6UML4ERjM+xCctQoEh Mf4tLpra2Kc11JY/N8uwZvsUDaRc0uJKpSQAIgk4JEcMjHVLKLyTmVTrIPWejmV1 YvQBt/NBdJA6mK7peezTLCuRQRRHSbiYgDOV2+n+EeczzhKKkl2+z0JwDCFH8QIM FCNiW97/ya+500s3SpVr0rbstMc0VgD6n+1z0AWa84w7h3ZXJK0= =cSXC -----END PGP SIGNATURE----- --QxkYVnuAhcV7vTIA1FrtnORyYqBLqkbFV-- From owner-freebsd-hackers@freebsd.org Fri Aug 30 12:44:22 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D479ACDF49; Fri, 30 Aug 2019 12:44:22 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46KfLZ51Cpz45XX; Fri, 30 Aug 2019 12:44:22 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id 5F47E3C0199; Fri, 30 Aug 2019 12:44:16 +0000 (UTC) Date: Fri, 30 Aug 2019 12:44:16 +0000 From: Brooks Davis To: Ed Maste Cc: "Rodney W. Grimes" , FreeBSD Hackers , fcp@freebsd.org Subject: Re: FCP 20190401-ci_policy: CI policy Message-ID: <20190830124416.GC19377@spindle.one-eyed-alien.net> References: <201908291905.x7TJ5Bw8091371@gndrsh.dnsmgr.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="vOmOzSkFvhd7u8Ms" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-Rspamd-Queue-Id: 46KfLZ51Cpz45XX X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.96 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.96)[-0.959,0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 12:44:22 -0000 --vOmOzSkFvhd7u8Ms Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Aug 29, 2019 at 08:05:06PM -0400, Ed Maste wrote: > On Thu, 29 Aug 2019 at 15:05, Rodney W. Grimes > wrote: > > > > Here in lies one of the fundemental problems, this view by some that > > a "revert commit is something bad, it is kind of punishment". That is > > not true. Reverts are GREAT things, they allow the tree to be returned > > to a known state, usually quicly. The original commit is STILL IN SVN, > > and a bad revert can guess what.. be reverted!. >=20 > Let me echo Rod here. I'm also very happy that this statement was made > by one of the original FreeBSD committers. >=20 > Reverting a change is not an insult, not a punishment, not something > bad - it's simply an acknowledgement that some aspect of the change > didn't meet expectations. We should be considerably more willing to revert changes that break things. I agree with Warner that it's worth pinging the developer in question, but if they don't respond in in a couple hours, I see no value in waiting further. At that point any loss of productivity is on them and they have already wasted considerable productivity for other committers. All that being said, the LLVM project's aggressive use of reverts is sometimes enormously costly to downstreams with large changes[0] and we should avoid swinging too far toward reverts. The issue comes about when a non-trivial patch conflicts with a local change. When such a patch is reverted that's another conflict and sometimes you end up making/unmaking the same fixes multiple times when a change causes problems in an environment that developer can't test directly. Some balance is required, but I think we should be considerably more willing to revert changes where a fix isn't immediately available. -- Brooks [0] On the CHERI project we're almost certainly the largest public downstream for LLVM to the point that they validated the git monorepo migration tools on our tree(s). --vOmOzSkFvhd7u8Ms Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJdaRofAAoJEKzQXbSebgfA7xwH/im4SHdyaGqJtJLmCRu2/HF4 3NyTBv91b3pPBYaukl61xFmMwjyZBeQS9OnxozVhyw5fpGjKCHPNEQzoaUzjEMHm 2iL3ag57Dj0hyNuVgJWyPSauSjozLBOMSrOLJLzs/VWq1R/+325YMUJ6nRxnNewF YhnOQiErnlFalWUB2QQ3Gg73sSaXdagLTE+EOlCNuFSBQ+ead3A6mOURspGKSj/u 5LHi6Jr4mOVv9hg2Qgt+sN25WbgGW0o/2UDGY7iNDkEJk0CkzYXN3o53V6pabQHJ 9VfTT0V3sf4xf61Bq/Bp0jM0cjBt0FoduoiJab3e9oM+dRRnUf910rpHf3w5S1s= =9/Vl -----END PGP SIGNATURE----- --vOmOzSkFvhd7u8Ms-- From owner-freebsd-hackers@freebsd.org Fri Aug 30 16:06:43 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E1653D1D76 for ; Fri, 30 Aug 2019 16:06:43 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46Kkr35gWxz4Fq6; Fri, 30 Aug 2019 16:06:43 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro-4.local (ralph.baldwin.cx [66.234.199.215]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 49D6976F5; Fri, 30 Aug 2019 16:06:43 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Subject: Re: FreeBSD Security Advisory FreeBSD-SA-19:23.midi To: Eugene Grosbein , Bruce Evans Cc: Freebsd hackers list References: <20190820201257.7A9D41F8B7@freefall.freebsd.org> <20190830133855.W1405@besplex.bde.org> <5961a5af-d36c-4b8f-20c1-e7054b0149f4@grosbein.net> From: John Baldwin Openpgp: preference=signencrypt Autocrypt: addr=jhb@FreeBSD.org; keydata= mQGiBETQ+XcRBADMFybiq69u+fJRy/0wzqTNS8jFfWaBTs5/OfcV7wWezVmf9sgwn8TW0Dk0 c9MBl0pz+H01dA2ZSGZ5fXlmFIsee1WEzqeJzpiwd/pejPgSzXB9ijbLHZ2/E0jhGBcVy5Yo /Tw5+U/+laeYKu2xb0XPvM0zMNls1ah5OnP9a6Ql6wCgupaoMySb7DXm2LHD1Z9jTsHcAQMD /1jzh2BoHriy/Q2s4KzzjVp/mQO5DSm2z14BvbQRcXU48oAosHA1u3Wrov6LfPY+0U1tG47X 1BGfnQH+rNAaH0livoSBQ0IPI/8WfIW7ub4qV6HYwWKVqkDkqwcpmGNDbz3gfaDht6nsie5Z pcuCcul4M9CW7Md6zzyvktjnbz61BADGDCopfZC4of0Z3Ka0u8Wik6UJOuqShBt1WcFS8ya1 oB4rc4tXfSHyMF63aPUBMxHR5DXeH+EO2edoSwViDMqWk1jTnYza51rbGY+pebLQOVOxAY7k do5Ordl3wklBPMVEPWoZ61SdbcjhHVwaC5zfiskcxj5wwXd2E9qYlBqRg7QeSm9obiBCYWxk d2luIDxqaGJARnJlZUJTRC5vcmc+iGAEExECACAFAkTQ+awCGwMGCwkIBwMCBBUCCAMEFgID AQIeAQIXgAAKCRBy3lIGd+N/BI6RAJ9S97fvbME+3hxzE3JUyUZ6vTewDACdE1stFuSfqMvM jomvZdYxIYyTUpC5Ag0ERND5ghAIAPwsO0B7BL+bz8sLlLoQktGxXwXQfS5cInvL17Dsgnr3 1AKa94j9EnXQyPEj7u0d+LmEe6CGEGDh1OcGFTMVrof2ZzkSy4+FkZwMKJpTiqeaShMh+Goj XlwIMDxyADYvBIg3eN5YdFKaPQpfgSqhT+7El7w+wSZZD8pPQuLAnie5iz9C8iKy4/cMSOrH YUK/tO+Nhw8Jjlw94Ik0T80iEhI2t+XBVjwdfjbq3HrJ0ehqdBwukyeJRYKmbn298KOFQVHO EVbHA4rF/37jzaMadK43FgJ0SAhPPF5l4l89z5oPu0b/+5e2inA3b8J3iGZxywjM+Csq1tqz hltEc7Q+E08AAwUIAL+15XH8bPbjNJdVyg2CMl10JNW2wWg2Q6qdljeaRqeR6zFus7EZTwtX sNzs5bP8y51PSUDJbeiy2RNCNKWFMndM22TZnk3GNG45nQd4OwYK0RZVrikalmJY5Q6m7Z16 4yrZgIXFdKj2t8F+x613/SJW1lIr9/bDp4U9tw0V1g3l2dFtD3p3ZrQ3hpoDtoK70ioIAjjH aIXIAcm3FGZFXy503DOA0KaTWwvOVdYCFLm3zWuSOmrX/GsEc7ovasOWwjPn878qVjbUKWwx Q4QkF4OhUV9zPtf9tDSAZ3x7QSwoKbCoRCZ/xbyTUPyQ1VvNy/mYrBcYlzHodsaqUDjHuW+I SQQYEQIACQUCRND5ggIbDAAKCRBy3lIGd+N/BCO8AJ9j1dWVQWxw/YdTbEyrRKOY8YZNwwCf afMAg8QvmOWnHx3wl8WslCaXaE8= Message-ID: Date: Fri, 30 Aug 2019 09:06:38 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <5961a5af-d36c-4b8f-20c1-e7054b0149f4@grosbein.net> Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 16:06:43 -0000 On 8/30/19 12:56 AM, Eugene Grosbein wrote: > 30.08.2019 11:03, Bruce Evans wrote: > >> The patch preserves some historical mistakes and adds some excessive >> verboseness about probe failures. I'm still waiting for jhb to reply to >> mails on 30 Oct 2015 and 23 Jan 2018 asking for a review of this patch >> or better a complete fix. > > Hmm... Maybe additional notification worth it :-) Hmm, I've used 'hint.foo.0.disabled=1' with many devices and it works fine. devctl enable can "undo" the disable post-boot even. It doesn't disable probing, only attaching once we know which driver a device is going to use. As far as I can tell, the patch makes it disable DEVICE_PROBE as well, but the vast majority of DEVICE_PROBE routines are idempotent. Only a handful of legacy ISA drivers use 'return (0)' to try to pass state from probe to attach via the softc. Those drivers could choose to check the disabled flag in their probe routine and/or be fixed to have idempotent probe routines. I think the latter is probably the best path forward. -- John Baldwin From owner-freebsd-hackers@freebsd.org Fri Aug 30 16:07:21 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E054CD1E70; Fri, 30 Aug 2019 16:07:21 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-qk1-x736.google.com (mail-qk1-x736.google.com [IPv6:2607:f8b0:4864:20::736]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46Kkrm1xwlz4FxG; Fri, 30 Aug 2019 16:07:20 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: by mail-qk1-x736.google.com with SMTP id w18so6628596qki.0; Fri, 30 Aug 2019 09:07:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=efKeDJjWAXaWx8IIAaUK8bYcGP3gvXT0u8vxDCpd0xc=; b=W0UqHRXFvbj8swQ8OqBD5cCOuruP6KrrnlnKfJaPL27NGWP8Le4GtGt5iGzJO4CbZp o4bX1jA8oSMUtW7s90ukwtJS5y7QnNwxYjqaqQpwxylofV0ZnrbfWp92+CeFZU2KIU96 584VFn5AQ4MJo19lDvL+fmfScCutfNvr8Of/WLkGyF47hnCh4rD0cJeRYdxk7QcFA3VL EXlbySzT23KOYpytorNu8BHBa6SofniKVnaxB+aD/1Rv2fNTi2993xNJpU/4jkATMIcE LSpMsg4OVaqmSu6HNrYSpztKSEq3IH3FR99JVtb/8ITgF4D4RcHyLY/PuXmh2C33ejeF P+dQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=efKeDJjWAXaWx8IIAaUK8bYcGP3gvXT0u8vxDCpd0xc=; b=ngBTWFYpG4KbGoUPBumWjGaJmKtn+ah11e7d1stT0hJ2441+WgApo15m5yt1sxeC8D FkaYMz8oHW7MEqKA+C88WU54V+TdheZcAMoevV7blOmh+YzyJIxRoOT5BUV3byqZLKl3 u/g6JR9bU/qTMKsn93IY1jZXzgbqDZZCbfy6wBt68f7l/T/N2THxDdLPQwCXM3q4RjRy F75+Jmu5yEEHbuv3QIVTe0Yt8lkpa4F9vbE+RQDO5+4kZ6dQFYED0ivtx8JVYRc9pY6b dKRcq23/DiamwLzPS6m+z88hcPy1IHjErId5JroDqCGj/GPl7tBbv25N4cH/vBgkvfr4 IUgQ== X-Gm-Message-State: APjAAAX/v42gQ/sZ7+JUUpf6bCgNBqhtVAyq1NyJmACsIwbDBxm47LQx /QYwqriCAQYHcHrM0AVLfY7xpkEo X-Google-Smtp-Source: APXvYqyHRw3cnRAwzyhptR+4E4X9LBcOY2YzjV+0/I7VGZGhlFrQuiH/16aXwHVq0Urq4MXcENdliw== X-Received: by 2002:ae9:e202:: with SMTP id c2mr15118816qkc.15.1567181239000; Fri, 30 Aug 2019 09:07:19 -0700 (PDT) Received: from [10.192.166.0] (stargate.chelsio.com. [12.32.117.8]) by smtp.googlemail.com with ESMTPSA id h29sm3615902qtb.46.2019.08.30.09.07.17 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 30 Aug 2019 09:07:18 -0700 (PDT) Subject: Re: hwpmc ktrdump: failed to resolve ktr symbols To: shreyank amartya , freebsd-drivers@freebsd.org Cc: freebsd-hackers@freebsd.org References: From: Navdeep Parhar Message-ID: <0534e1a9-af57-96cf-9ea2-ec4b30021226@gmail.com> Date: Fri, 30 Aug 2019 09:07:16 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46Kkrm1xwlz4FxG X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=W0UqHRXF; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of nparhar@gmail.com designates 2607:f8b0:4864:20::736 as permitted sender) smtp.mailfrom=nparhar@gmail.com X-Spamd-Result: default: False [-3.99 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.99)[-0.992,0]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (-9.38), ipnet: 2607:f8b0::/32(-2.84), asn: 15169(-2.32), country: US(-0.05)]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[6.3.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 16:07:21 -0000 On 2019-08-30 03:54, shreyank amartya wrote: > Hi, > > I'm trying to collect traces from hwpmc module. I compiled the module with > -DHWPMC_DEBUG option and set the sysctl kern.hwpmc.debugflags but I'm > unable to retrive the traces using ktrdump. I get this error message: > > ktrdump: failed to resolve ktr symbols > > I'm guessing that this error occurs because the symbols for hwpmc module > (compiled and loaded separately) are not a part of kernel. If this is the > case, then how do I get those symbols. In the ktr man page its also > mentioned that ddb and gdb can retrieve ktraces. > > I found the ddb command show ktr. But how do I invoke ddb (without a panic)? sysctl debug.kdb.enter=1 > > I'm unable to find how can I use gdb to view the traces (I launched kgdb > but could not find a command to show kernel traces). > > Any idea on how to get ktrdump working? Or get the kernel traces. You need a kernel with KTR support. See ktr(4) for a full list of the options and other details. See "sysctl kern.conftxt" for what's in your running kernel. Regards, Navdeep From owner-freebsd-hackers@freebsd.org Fri Aug 30 16:19:55 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 89942D243B; Fri, 30 Aug 2019 16:19:55 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46Kl7G1Hmrz4GV8; Fri, 30 Aug 2019 16:19:53 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x7UGJnOn095329; Fri, 30 Aug 2019 09:19:49 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x7UGJnUt095328; Fri, 30 Aug 2019 09:19:49 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201908301619.x7UGJnUt095328@gndrsh.dnsmgr.net> Subject: Re: FCP 20190401-ci_policy: CI policy In-Reply-To: <20190830015805.GA16894@troutmask.apl.washington.edu> To: sgk@troutmask.apl.washington.edu Date: Fri, 30 Aug 2019 09:19:49 -0700 (PDT) CC: Cy Schubert , Ian Lepore , FreeBSD Hackers , araujo@freebsd.org, fcp@freebsd.org, Konstantin Belousov , Li-Wen Hsu , Kristof Provost X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 46Kl7G1Hmrz4GV8 X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of freebsd-rwg@gndrsh.dnsmgr.net has no SPF policy when checking 69.59.192.140) smtp.mailfrom=freebsd-rwg@gndrsh.dnsmgr.net X-Spamd-Result: default: False [2.43 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.49)[0.485,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[dnsmgr.net]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.77)[0.767,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.23)[0.233,0]; RCPT_COUNT_SEVEN(0.00)[9]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.04)[ip: (0.15), ipnet: 69.59.192.0/19(0.07), asn: 13868(0.05), country: US(-0.05)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 16:19:55 -0000 > On Thu, Aug 29, 2019 at 06:35:41PM -0700, Cy Schubert wrote: > > > > What about weekends? I notice commits tend to slow down during weekends. I > > suspect people who work on FreeBSD at $COMPANY for a living treat it as a > > job, which it is. However this should be considered. > > > > A committer, who works for a $COMPANY and has no intention of > looking at the mailing lists or CI reports until Monday, should > not commit on Firday. What ever happen to common sense? Perhaps this speaks to another issue then, I know the community is a whole against rules, procedures, requirements, etc all, but I'll again stick my head across the block and say: Should it be added to the committers guide that a committer is expected to be "responsive" to commit especially, and project in general emails for 48 hours following any commit? Would that not solve some of this? I do know from first hand interactions that several project members foo@freebsd.org email is basically /dev/nulled into a mail box that gets cleaned out on some random schedule. That technically goes against the requirement that if your a committer your suppose to be reading commit mail but I can agree that the volume of commit mail has become an overwhelming problem. One I do wish we had a better answer to than just ignore it. Regards, -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-hackers@freebsd.org Fri Aug 30 16:25:27 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9FA6ED271F; Fri, 30 Aug 2019 16:25:27 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pl1-x629.google.com (mail-pl1-x629.google.com [IPv6:2607:f8b0:4864:20::629]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KlFf4Rxvz4GtX; Fri, 30 Aug 2019 16:25:26 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mail-pl1-x629.google.com with SMTP id o3so3563526plb.13; Fri, 30 Aug 2019 09:25:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=qYmEuKmi7qTb6I3uIiGtbUztIC0f+a/EYr6bJdfUz1w=; b=YwcqkjhFPMFasBJ74QpHadWVIBvCy0GMMmLfVHu/jUlCq7zdwso4js3ge4ePv6iSmi k3WD/08DgzQp+X7FvNxQBmIOyQwxgSrPwuNjC8UEtxjvOLn1kCCvuB7x9+HFnqzODtj9 mVdCVRL9WdTsWu2ipFgOGpJDE6SWOPpZQKHTkFsJeEoNQosiMH+Ju8ZjsAp5OtTkYBSd 3Dd4VWsF+W/LTpaE1dkU8/830DACrT30fY7MSqEujqVMUiVJRA4kE7+XXawc7dLO5awB n/IvcobtmlSUY0GyCseJacjJxg86/z+aInqGm/fCUYU+gxk41EIQI4iivdwVwunEg7ah BucA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=qYmEuKmi7qTb6I3uIiGtbUztIC0f+a/EYr6bJdfUz1w=; b=tVIClHc8A2zNmCCExNsMdYeUGREtw6XbW8CWBITA4nQsL9YxQEc75CueK49JHu8hfA 1yB+jCc39sysPubTLtRzBZVIjBHmU7QO9I7ZElvTRj0Tp1aKpLoqYqQGIpukBQlOQGcZ BIDpi2JZASKI2xIe36jpn5jeXxwUb70L/dyMMuMCdELjNbO7GCf/VvTlQ1JU65CrRw3N JaXZL7sLd0VTFK1JjVhcNUnYf3hoOZ+okykqIlQKAf29GuVToKjzFDgwRN4leUDJM0IN 5pnWqnlTZOcBajliQajeW8tVAixjRT7OTXAXMMVhNxSozrEzpTJ4WQpFDdaXIgGCWcgd SOZA== X-Gm-Message-State: APjAAAWH8dXRvgNxkAfGfueBI9X7a+NUc3UFMI7LDV24nTM+gdsJUfeR uiWIN/o6Yi2NOv9G49f3/rTUb7SX X-Google-Smtp-Source: APXvYqz6tZcn1sJ82mD87PDMxRpzEjXbQnmWf7e2Mp2b8Jx6KcTB4cCnFshavJiFz92bhh6Ca1I8Eg== X-Received: by 2002:a17:902:fe0f:: with SMTP id g15mr16295103plj.2.1567182324414; Fri, 30 Aug 2019 09:25:24 -0700 (PDT) Received: from ?IPv6:2607:fb90:b23a:cc00:d046:1756:4288:91f6? ([2607:fb90:b23a:cc00:d046:1756:4288:91f6]) by smtp.gmail.com with ESMTPSA id a5sm5274066pjs.31.2019.08.30.09.25.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 30 Aug 2019 09:25:23 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (1.0) Subject: Re: FCP 20190401-ci_policy: CI policy From: Enji Cooper X-Mailer: iPhone Mail (16G77) In-Reply-To: Date: Fri, 30 Aug 2019 09:25:22 -0700 Cc: fcp@freebsd.org, FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: <339B7A20-F88D-4F60-B133-612189663272@gmail.com> References: To: Li-Wen Hsu X-Rspamd-Queue-Id: 46KlFf4Rxvz4GtX X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=YwcqkjhF; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of yaneurabeya@gmail.com designates 2607:f8b0:4864:20::629 as permitted sender) smtp.mailfrom=yaneurabeya@gmail.com X-Spamd-Result: default: False [-3.49 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.99)[-0.989,0]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (-9.17), ipnet: 2607:f8b0::/32(-2.84), asn: 15169(-2.32), country: US(-0.05)]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE_FREEMAIL(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[9.2.6.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 16:25:27 -0000 > On Aug 27, 2019, at 21:29, Li-Wen Hsu wrote: >=20 > It seems I was doing wrong that just changed the content of this FCP > to "feedback", but did not send to the right mailing lists. >=20 > So I would like to make an announcement that the FCP > 20190401-ci_policy "CI policy": >=20 > https://github.com/freebsd/fcp/blob/master/fcp-20190401-ci_policy.md >=20 > is officially in "feedback" state to hopefully receive more comments > and suggestions, then we can move on for the next FCP state. First off, thank you Li-Wen and Kristof for spearheading this proposal; it=E2= =80=99s a very contentious topic with a lot of strong emotions associated wi= th it. As the person who has integrated a number of tests and helped manage them fo= r a few years (along with some of the care and feeding associated with them)= , this task is non-trivial. In particular when issues that I filed in bugzil= la are fixed quickly and linger in the tree for some time, impacting a lot o= f folks who might rely on build and test suite stability. The issue, as I see it, from a CI/release perspective that the new policy at= tempts to define a notion of =E2=80=9Cstable=E2=80=9D, in terms of both test= s and other code; right now, stability is sort of defined on a honor system b= asis with the FreeBSD test suite as a litmus test of sorts to convey a sense= of stability. =3D=3D=3D=3D=3D=3D One thing that I don=E2=80=99t see in the proposal is the health of the =E2=80= =9Cmake tinderbox=E2=80=9D target in a CI world (this is a gap in our curren= t CI process). Another thing that I don=E2=80=99t see in the proposal is about the health o= f head vs stable and how it relates to MFCs. I see a lot more issues occur o= n stable branches go unfixed for some time, in part because some fixes or en= hancements haven=E2=80=99t been MFCed. Part of the problem I see these days i= s a bit of a human/resource problem: if developers can=E2=80=99t test their c= hanges easily, they don=E2=80=99t MFC them. This issue has caused me to do a fair amount of triage in the past when back= porting changes, in order to discover potentially missing puzzle pieces in o= rder to make my tests and code work. =3D=3D=3D=3D=3D=3D The big issues, as I see it based on the discussions that has taken place in= the thread, is around revert timing and etiquette, and dealing with unrelia= ble tests. First off, revert timing and etiquette: while I see the FCP as an initial fr= amework, I am a bit concerned with the heavy handed ness of =E2=80=9Cwhat co= nstitutes needing reversion=E2=80=9D: should this be done after N consistent= failures in a certain period (be they build or test)? Furthermore, why is a= human involved in making this decision (apart from maybe a technical soluti= on via automation not being available yet)? Second off, unreliable tests: * Unreliable tests need to be qualified not based on a single run, but a pat= tern of runs. The way that this worked at Facebook is, if a test failed, it would attempt t= o rerun it multiple times (10 in total IIRC). If the test was consistently f= ailing on a build, the test would be automatically disabled, and all committ= ers in a revision range would be nagged as part of disabling those tests. Th= is generally works because of siloization of Facebook components, but is a m= uch harder problem to solve with FreeBSD because it is a complete OS distribution and sometimes small seemingly disconnected changes can cause= a lot of grief. So what to do? I suggest expanding the executors and running individuals suites instead of t= he whole batch of tests. While it wouldn=E2=80=99t fix everything and would b= e an expensive thing to do with our current test infrastructure, it would al= low folks to better pinpoint issues and be able to get some level of coverag= e, as opposed to throwing all of test execution out, like a baby with the ba= th water. How do we get there? - Expand the CI executor pool. - Provide a tool or process with which we can define test suites. - Make spinning up executors faster: with virtual machines this is typically= done by using Big Iron infrastructure clusters (e.g., ESXi clusters) and so= mething like thin provisioning where one could start from a common image/sna= pshot, instead of taking a hit copying around images. Linux can do this with= btrfs; we can do this with ZFS with per VM datasets, snapshotting, etc. While this only gets part of the way to a potential solution, it is a good w= ay to begin solving the isolation/execution problem. * A number of tests that existed in the tree have varying quality/reliabilit= y; I agree that system level tests (of which the pf tests are one of many) a= re less reliable than unit/API functional tests. This is the nature of the b= east of testing. The core issue I see with the test suite as it stands, is that it mixes inte= gration/system level tests (less deterministic) with functional/unit tests (= generally more deterministic). Using test mock frameworks would be a good technical solution to making syst= em tests into functional/unit tests (googlemock and unittest.mock are two of= many good tools I know of in this area), but we need a way to run both case= s. I can see now where some of the concern over labeling test types was a conce= rn when I first started this work (des@/phk@ aired this concern). Part of the technical/procedural solution to allowing commingling of tests i= s to go back and label the tests appropriately. I=E2=80=99ll send out an FCP= for this sometime in the next week or two. =3D=3D=3D=3D=3D=3D Taking a step back, as others have brought up, we=E2=80=99re currently hinde= red by tooling: we are applying a DVCS (git, hg) based technique (CI) to sub= version and testing changes after they=E2=80=99ve hit head, instead of befor= e they hit head. While phabricator can partially solve this by testing upfront (we don=E2=80=99= t enforce this; I=E2=80=99ve made my concerns with this not being a requirem= ent well-known in the past), the solution is limited by bandwidth for testin= g, i.e., testing is an all or nothing exercise right now and building multip= le toolchains/architectures takes a considerable amount of time. We could le= verage cloud/distributed solutions for this (Cirrus CI, Travis if the integr= ation existed), but this would require using github or teaching a tool how t= o make the appropriate REST api calls to run the tests and query the status (= in progress, pass, fail, etc). Applying labels and filtering on test suites will get us partway to a final s= olution from a test perspective, but a lot of work needs to be done with pha= bricator, etc. We also need to have build failures with tier 1 architectures with GENERIC b= e a commit blocking operation. Full stop. =3D=3D=3D=3D=3D=3D While some of the thoughts I put down aren=E2=80=99t complete solutions, I h= ave subproposals that should be done/things that could be worked on before i= mplementing the proposed CI policy. Some of the things I brought up above=20= While I can=E2=80=99t work on it now, December break is coming up, and with i= t I=E2=80=99ll have more time to work on projects like this. I=E2=80=99ll pu= t down some TODO items so I can look at tackling them during the break. Thank you, -Enji= From owner-freebsd-hackers@freebsd.org Fri Aug 30 17:16:26 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 78C4BD3B56 for ; Fri, 30 Aug 2019 17:16:26 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x72a.google.com (mail-qk1-x72a.google.com [IPv6:2607:f8b0:4864:20::72a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KmNT3t23z4Kn4 for ; Fri, 30 Aug 2019 17:16:25 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x72a.google.com with SMTP id u190so6808189qkh.5 for ; Fri, 30 Aug 2019 10:16:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=uNnN80wf4y3qhRKTyWGPo/Cnl/OLMVpK+PlukUbmGgo=; b=0qzaDiLOqg+Yk4q96U4MrrKNCGzO/8mMqcYprU2LhNCccsrBvmO7tBOlVwdE+jZhZp aKChYAEWCPY34rKRLnL0sMlSOFKsAkPHL+7EJNGhdnjUqyQA+ii7+d85ve9UgiP657/8 VoHNZiigf9kFP3wj09GUZ8zbhk1PEbEZdrr7VkQWoqSrBq53fbMOM4H86XsS7+RIj9k4 sTc2mDHGOtAsBQjoJS9XOhgLdLL5xpiIl8yZw8QRffkBMpcVw7PvWAdNukZNns6MILHl XN31DpiWX/orDtijKrEZXkpHdQja3FZT8onm9DniaRAbdC7vpCNw+zOHzPVIjpx6drKN 6r6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=uNnN80wf4y3qhRKTyWGPo/Cnl/OLMVpK+PlukUbmGgo=; b=H+oqTfJ7du+7hrn7pCMeTXBbJN+Db+OaUwQsvq06T4Us/fWHHi1tW+nzaTlWI1OLYI Vox6quI26abm1tmr++mRIHzjMq/gQGzevuxsRcCTLKFVOC4wbaf66o+fVpWbilugwB+R I4UQbAlLCLTrAiKpYzMn3+GhTZJbqy9rHZ2/O5apleH3Yc4bjF/bf+XiN+UDlN4xrA9e P6PFmUTT+4Wk2IKA20WX/cqg2MIbkj7ZuqM6AtxIPbbFuw+keCd80IM5r3rrOYhWbYse fbZw2xpVnv9qXuJgDnOx5L+5ZzKxFfLVI3+HqtoyCAzidMzzOXHAJUhYhg9BKjQ4r3mB Z1Cw== X-Gm-Message-State: APjAAAVgJ0SsM/c1G9gRPFFgWgz+kEFHK36cow3LRaa+BRZ9GvEPyhth 4p/ss8SYjoSg60ieJmBZPAK+6CxW3aFsdsAZ1T23oQ== X-Google-Smtp-Source: APXvYqwS1MXThCdK17aTku4YhtTJn3v0mEhqZj87Y9OeHg/PDlr6Ie0YOo4Og3CzDTqoQXuF7y6fdSKTHw5HfDd91As= X-Received: by 2002:a37:2c07:: with SMTP id s7mr16228481qkh.495.1567185384081; Fri, 30 Aug 2019 10:16:24 -0700 (PDT) MIME-Version: 1.0 References: <20190820201257.7A9D41F8B7@freefall.freebsd.org> <20190830133855.W1405@besplex.bde.org> <5961a5af-d36c-4b8f-20c1-e7054b0149f4@grosbein.net> In-Reply-To: From: Warner Losh Date: Fri, 30 Aug 2019 11:16:12 -0600 Message-ID: Subject: Re: FreeBSD Security Advisory FreeBSD-SA-19:23.midi To: John Baldwin Cc: Eugene Grosbein , Bruce Evans , Freebsd hackers list X-Rspamd-Queue-Id: 46KmNT3t23z4Kn4 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=0qzaDiLO; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::72a) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-5.91 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-0.996,0]; RCVD_IN_DNSWL_NONE(0.00)[a.2.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.92)[ip: (-9.37), ipnet: 2607:f8b0::/32(-2.84), asn: 15169(-2.32), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 17:16:26 -0000 On Fri, Aug 30, 2019 at 10:06 AM John Baldwin wrote: > On 8/30/19 12:56 AM, Eugene Grosbein wrote: > > 30.08.2019 11:03, Bruce Evans wrote: > > > >> The patch preserves some historical mistakes and adds some excessive > >> verboseness about probe failures. I'm still waiting for jhb to reply to > >> mails on 30 Oct 2015 and 23 Jan 2018 asking for a review of this patch > >> or better a complete fix. > > > > Hmm... Maybe additional notification worth it :-) > > Hmm, I've used 'hint.foo.0.disabled=1' with many devices and it works fine. > devctl enable can "undo" the disable post-boot even. > > It doesn't disable probing, only attaching once we know which driver a > device > is going to use. As far as I can tell, the patch makes it disable > DEVICE_PROBE as well, but the vast majority of DEVICE_PROBE routines are > idempotent. Only a handful of legacy ISA drivers use 'return (0)' to try > to > pass state from probe to attach via the softc. Those drivers could choose > to > check the disabled flag in their probe routine and/or be fixed to have > idempotent probe routines. I think the latter is probably the best path > forward. > The problem with disabling before PROBE is that if you have N foo devices, hint.foo.0.disabled=1 will disable all of them as the probe routines all return 'not me' and we try foo0 on each new instance... I'm pretty sure that's why it's not done today and why I didn't do it when I added this feature because how do you know you have foo0 until probe says 'yup, that's mine'? Most of the old ISA routines that returned 0 I think have been deleted. Maybe all since they were for fussy hardware from before the plug and play era... Warner From owner-freebsd-hackers@freebsd.org Fri Aug 30 17:42:32 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3B925D4672 for ; Fri, 30 Aug 2019 17:42:32 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x734.google.com (mail-qk1-x734.google.com [IPv6:2607:f8b0:4864:20::734]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46Kmyb36YNz4MKc for ; Fri, 30 Aug 2019 17:42:31 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x734.google.com with SMTP id f10so6866209qkg.7 for ; Fri, 30 Aug 2019 10:42:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=mEDmYyGD+aYCJIMJYKGfdtezYOt6DpNDj2A2wQiDr08=; b=VUzHJ6gChxj7MjPGyoI5/keny6o1rjNfuJHVGRmcrGI0hYpSo4b23PWanSGJHUDYhR /IpvuKffI5ofRjssqynombCtRpQanlYw5kT3tWUny5XnlQySBBy7rI+4CisHmVPnouN0 J51YyROOu5TuulfIBZ350CJGWTROspnalzGsKnluCavCkWS17UQDX1dnzv3V8I4I3Tm2 VVm3UYx7KnhEEcc8e+l9aqOax3G9xNWdyEd4yUVxqfEBPukCY1DnQ4GYLTUo3AiPpauG EjAMoXooXgiiANRSb4wLoY6/IiEt7sCx/7IEP4y2mnuYEgleDkBObIg0NiNdC7qRQZjy Hq5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=mEDmYyGD+aYCJIMJYKGfdtezYOt6DpNDj2A2wQiDr08=; b=JKOtlq0eaf54kSGM91XTs9QMKyY+//wTKTjn9DlpP/oW3yQB8ZlFDSj3e0vomlTudB Y+usFtktuLs+r/ze02trAh6TkWOe6TPLprwyGgDcaLCfAnhK1ahPToR7PD9C+iU/hNtH OLt/6lQMgR/0kMYWbijSAOPL1Z45poQETdVUtrKJ7VlFs9EW9uYcN40/yrXhGMaEXydl qDzKVm7ftuINZEr7t1ZwbOoGjLl7xJywhcgu6F44nNMY5lbPcS4SOa7kH2c4XKmPmn+u rdfMCGXvA9xIa1oA7S2hDDtnSe1eiziD8ESRUsvKlqnPb7y4jmsGeg/js/gUnUTG+iS3 Zdaw== X-Gm-Message-State: APjAAAX9ZdCNwoyZpKDVCwjHzK9q173N5MhHspQKx7U3evcBAaST+CIk uzp5Q3ZJ/UNg199eiejIxqlOYJzcJ2lvQ0w2NZGbpkKZD7g= X-Google-Smtp-Source: APXvYqx2MNSmSeQi3YG5eOV6OO7gtnz8xJKpvOafPnEie0JxjyLF+yTqI3YbtFyr3vqifFE5iFvwr+tVqTyprh1YGOY= X-Received: by 2002:a37:4804:: with SMTP id v4mr17147516qka.60.1567186950326; Fri, 30 Aug 2019 10:42:30 -0700 (PDT) MIME-Version: 1.0 References: <339B7A20-F88D-4F60-B133-612189663272@gmail.com> In-Reply-To: <339B7A20-F88D-4F60-B133-612189663272@gmail.com> From: Warner Losh Date: Fri, 30 Aug 2019 11:42:19 -0600 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Enji Cooper Cc: Li-Wen Hsu , FreeBSD Hackers , fcp@freebsd.org X-Rspamd-Queue-Id: 46Kmyb36YNz4MKc X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=VUzHJ6gC; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::734) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-5.92 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-0.996,0]; RCVD_IN_DNSWL_NONE(0.00)[4.3.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; FREEMAIL_TO(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.92)[ip: (-9.41), ipnet: 2607:f8b0::/32(-2.84), asn: 15169(-2.32), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 17:42:32 -0000 On Fri, Aug 30, 2019 at 10:25 AM Enji Cooper wrote: > > Taking a step back, as others have brought up, we=E2=80=99re currently hi= ndered by > tooling: we are applying a DVCS (git, hg) based technique (CI) to > subversion and testing changes after they=E2=80=99ve hit head, instead of= before > they hit head. > > While phabricator can partially solve this by testing upfront (we don=E2= =80=99t > enforce this; I=E2=80=99ve made my concerns with this not being a require= ment > well-known in the past), the solution is limited by bandwidth for testing= , > i.e., testing is an all or nothing exercise right now and building multip= le > toolchains/architectures takes a considerable amount of time. We could > leverage cloud/distributed solutions for this (Cirrus CI, Travis if the > integration existed), but this would require using github or teaching a > tool how to make the appropriate REST api calls to run the tests and quer= y > the status (in progress, pass, fail, etc). > > Applying labels and filtering on test suites will get us partway to a > final solution from a test perspective, but a lot of work needs to be don= e > with phabricator, etc. > > We also need to have build failures with tier 1 architectures with GENERI= C > be a commit blocking operation. Full stop. > Until we have a DCVS, this is a non-starter. It's too hard with svn. Let's table it until we have the migration to git complete. Also FreeBSD is huge, having to wait for a full build on all Tier-1 platforms also is a non-starter. It takes too long, and there's too many ways to build FreeBSD. Having a sanity check incremental build may be OK, but there's a number of changes that break the incremental -DNO_CLEAN build that are none-the-less fine for a complete rebuild. There's a ton of details to get right here to make it not an absolute nightmare for developers to get patches in and slow the velocity of changes to a crawl. Since we know nothing of our future git overlords, it's premature to even start this discussion because so many things dovetail with that effort we won't get beyond the basics (which people generally agree in principle on, but have concerns about the details that will be filibustered to death in the absence of a concrete git system it will add-on to). tl;dr: Love the concept, the devil is in the details to ensure we don't stifle momentum. Warner From owner-freebsd-hackers@freebsd.org Sat Aug 31 01:55:01 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 62E5BDF9AE; Sat, 31 Aug 2019 01:55:01 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-io1-f45.google.com (mail-io1-f45.google.com [209.85.166.45]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46Kztr2MW6z3LR7; Sat, 31 Aug 2019 01:55:00 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-io1-f45.google.com with SMTP id x4so17745216iog.13; Fri, 30 Aug 2019 18:55:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ZIfZggj9toIo5ZvUWoDtnDPGgIY3F/IW8VfjLo7w8Gk=; b=GCDU0An6agiz9uw1S1atc+Wli4Ct/X0pfjieenZUD/eenyjSDhsYi1qu9m12caXDmJ AcAZtMLu6xApae4iP0Kt4f3tbpws5erzrHLlmiafY5IN9DDR9n+swqkS7IRRzEc0lv0S pZBJt0TmePhTSrWuNRcGU5aJgnJKI8Dd1iBD9xGsOgxnjOUIGpE8bte8ZpMXuu+j+Dk/ 4WXhcnQxPvgU4F5WaFc11rVHKWLKHQ8IVqX6qzaX8PlvBrkj822iomrUA5YkfelfNyDg GWTM4IR5Ckaq+GgU+eC34ZWQFzOPq6cKhq/4vjUjxKfmH99oOpw2Y5Zf4BxkQp7+fpqq akLA== X-Gm-Message-State: APjAAAVDvd6oXC0kN/jbpEt+C9meEzHantPIWlv8mZr37EqNBwfZEuLl 5Vj20drrVpgYbENMJvibFa8w2H+9Qcx6cJWjmWMcfw== X-Google-Smtp-Source: APXvYqw/OIRsCk3ey0qml76Ht6ZFPpqKJcRrzpsDIHUUj32U3B+etIvX/eJpRxDfFnOFSZD5UC+zwS4tOoGKcmmdvrE= X-Received: by 2002:a5e:881a:: with SMTP id l26mr1460058ioj.185.1567216498988; Fri, 30 Aug 2019 18:54:58 -0700 (PDT) MIME-Version: 1.0 References: <20190829114057.GZ71821@kib.kiev.ua> <20190830065534.GC71821@kib.kiev.ua> In-Reply-To: <20190830065534.GC71821@kib.kiev.ua> From: Ed Maste Date: Fri, 30 Aug 2019 21:54:41 -0400 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Konstantin Belousov Cc: Li-Wen Hsu , FreeBSD Hackers , fcp@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46Kztr2MW6z3LR7 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.45 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-5.27 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.991,0]; RCVD_IN_DNSWL_NONE(0.00)[45.166.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.28)[ip: (-5.67), ipnet: 209.85.128.0/17(-3.34), asn: 15169(-2.31), country: US(-0.05)]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; FREEMAIL_TO(0.00)[gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Aug 2019 01:55:01 -0000 On Fri, 30 Aug 2019 at 02:56, Konstantin Belousov wrote: > > When I was (forced to) look into test failures, it was 50 vs. 50 % > of test bugs vs. some legitimately catched issues. Certainly if 50% of reported failures are actually test problems that's much too high. But independent of that, this still suggests the tests were responsible for reporting a good number of issues in advance of developers or end users. > > - The test is difficult to maintain > This is too. My main complain is that to debug a test case, I must strip > all atf* to be able to examine it under a debugger. Yes, this is my biggest complaint about our current test setup. But this impacts the tests' friendliness to developers, but not their efficacy in reporting regressions. From owner-freebsd-hackers@freebsd.org Sat Aug 31 08:55:16 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 00158CB22A; Sat, 31 Aug 2019 08:55:15 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46L9Cl5Jyvz4GbR; Sat, 31 Aug 2019 08:55:15 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x7V8t02i005839 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 31 Aug 2019 11:55:03 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x7V8t02i005839 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x7V8t0rf005838; Sat, 31 Aug 2019 11:55:00 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 31 Aug 2019 11:55:00 +0300 From: Konstantin Belousov To: Ed Maste Cc: Li-Wen Hsu , FreeBSD Hackers , fcp@freebsd.org Subject: Re: FCP 20190401-ci_policy: CI policy Message-ID: <20190831085500.GF71821@kib.kiev.ua> References: <20190829114057.GZ71821@kib.kiev.ua> <20190830065534.GC71821@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-Rspamd-Queue-Id: 46L9Cl5Jyvz4GbR X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.90 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.91)[-0.906,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Aug 2019 08:55:16 -0000 On Fri, Aug 30, 2019 at 09:54:41PM -0400, Ed Maste wrote: > On Fri, 30 Aug 2019 at 02:56, Konstantin Belousov wrote: > > > > When I was (forced to) look into test failures, it was 50 vs. 50 % > > of test bugs vs. some legitimately catched issues. > > Certainly if 50% of reported failures are actually test problems > that's much too high. 50% of what I looked at. The sample size was around 10. > But independent of that, this still suggests the > tests were responsible for reporting a good number of issues in > advance of developers or end users. > > > > - The test is difficult to maintain > > This is too. My main complain is that to debug a test case, I must strip > > all atf* to be able to examine it under a debugger. > > Yes, this is my biggest complaint about our current test setup. But > this impacts the tests' friendliness to developers, but not their > efficacy in reporting regressions. From owner-freebsd-hackers@freebsd.org Sat Aug 31 14:54:27 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0788ED3D4D; Sat, 31 Aug 2019 14:54:27 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from mx0a-00273201.pphosted.com (mx0a-00273201.pphosted.com [208.84.65.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.pphosted.com", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46LKBB5rXTz4YlV; Sat, 31 Aug 2019 14:54:26 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from pps.filterd (m0108158.ppops.net [127.0.0.1]) by mx0a-00273201.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id x7VEsO0V018357; Sat, 31 Aug 2019 07:54:24 -0700 Received: from nam02-bl2-obe.outbound.protection.outlook.com (mail-bl2nam02lp2051.outbound.protection.outlook.com [104.47.38.51]) by mx0a-00273201.pphosted.com with ESMTP id 2uqmkq8dvg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 31 Aug 2019 07:54:24 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Do0TjvybFu5WsE+DMB8AHNsYtFYiASK+sToCaFvFTGr1snY9WqgpmKOT1jX0L2DFW7L+7tkjZxR8k2F83VrG1W+Kp24nnV128oqzYeW81K8H6LSawLCdf8CbzCP8sy19mwyPH3RBAojJVrSUrOcFMnC4YcdPY1IwkXxgnVGvshTEFi5IvOqHgSEhq1u4dMoTG333rONOuJaZWJbSxLHyyarVvXxKL6lP409w/OYEPyk/8M16Eb96Nn+eS7tacYPn6FLVil5tMYnFFiKFYk9ylfBVO2726qsRVtl1NE+XdQjHERCmLKZsv+QBk52cY3tt0UZ//BBQhJW+/hX/EW3v9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JOI47K+2zg0zrn1pD9EAZcRd+qqbszg8o1IKtPLoWXY=; b=SxE4qQzGgZnRvKc1TWYGBylfLTmq4NeGPty38pfDF07BC+VP+eDOXiInBucnqFF/2bODIPDQ9G/uRSSsqDDC+e8Sn/asfchErGUiJ10WiC2YPjaNTxqpivz5/ZsJVd8We+LoYnrh0W4q30DILETDhuOndktHpW8otxlUvfuNOeb6GWHSMj/GZQiyKddbsyN2GilZaUb43Qd5urfe4iQ8lEJXMWT4TeykHOwTz39n4VxQa453IBYxrh7fvoiPoW8+wEGWmU0dkCcedU1j61mPrf4bG5W9Ua9ZYOgztQMQDVn1LiGRI12MVfk1hqij/xdX5MMg2d/7iBoXQK1RPAuM/w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=softfail (sender ip is 66.129.239.13) smtp.rcpttodomain=gmail.com smtp.mailfrom=juniper.net; dmarc=fail (p=reject sp=reject pct=100) action=oreject header.from=juniper.net; dkim=none (message not signed); arc=none Received: from BN3PR05CA0028.namprd05.prod.outlook.com (2603:10b6:400::38) by BL0PR05MB4692.namprd05.prod.outlook.com (2603:10b6:208:29::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2241.4; Sat, 31 Aug 2019 14:54:21 +0000 Received: from DM3NAM05FT013.eop-nam05.prod.protection.outlook.com (2a01:111:f400:7e51::203) by BN3PR05CA0028.outlook.office365.com (2603:10b6:400::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2241.7 via Frontend Transport; Sat, 31 Aug 2019 14:54:21 +0000 Received-SPF: SoftFail (protection.outlook.com: domain of transitioning juniper.net discourages use of 66.129.239.13 as permitted sender) Received: from P-EXFEND-EQX-02.jnpr.net (66.129.239.13) by DM3NAM05FT013.mail.protection.outlook.com (10.152.98.122) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2241.7 via Frontend Transport; Sat, 31 Aug 2019 14:54:19 +0000 Received: from P-EXBEND-EQX-01.jnpr.net (10.104.8.52) by P-EXFEND-EQX-02.jnpr.net (10.104.8.55) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sat, 31 Aug 2019 07:54:19 -0700 Received: from P-EXBEND-EQX-02.jnpr.net (10.104.8.53) by P-EXBEND-EQX-01.jnpr.net (10.104.8.52) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sat, 31 Aug 2019 07:54:18 -0700 Received: from p-mailhub01.juniper.net (10.104.20.6) by P-EXBEND-EQX-02.jnpr.net (10.104.8.53) with Microsoft SMTP Server (TLS) id 15.0.1367.3 via Frontend Transport; Sat, 31 Aug 2019 07:54:18 -0700 Received: from kaos.jnpr.net (kaos.jnpr.net [172.23.50.162]) by p-mailhub01.juniper.net (8.14.4/8.11.3) with ESMTP id x7VEsIiH018044; Sat, 31 Aug 2019 07:54:18 -0700 (envelope-from sjg@juniper.net) Received: by kaos.jnpr.net (Postfix, from userid 1377) id 5CA113F79D; Sat, 31 Aug 2019 07:54:18 -0700 (PDT) Received: from kaos.jnpr.net (localhost [127.0.0.1]) by kaos.jnpr.net (Postfix) with ESMTP id 5AE593F79C; Sat, 31 Aug 2019 07:54:18 -0700 (PDT) To: Ed Maste CC: Konstantin Belousov , FreeBSD Hackers , Li-Wen Hsu , , Subject: Re: FCP 20190401-ci_policy: CI policy In-Reply-To: References: <20190829114057.GZ71821@kib.kiev.ua> <20190830065534.GC71821@kib.kiev.ua> Comments: In-reply-to: Ed Maste message dated "Fri, 30 Aug 2019 21:54:41 -0400." From: "Simon J. Gerraty" X-Mailer: MH-E 8.6+git; nmh 1.7.1; GNU Emacs 26.1 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <70179.1567263258.1@kaos.jnpr.net> Date: Sat, 31 Aug 2019 07:54:18 -0700 Message-ID: <73111.1567263258@kaos.jnpr.net> X-EXCLAIMER-MD-CONFIG: e3cb0ff2-54e7-4646-8a04-0dae4ac7b136 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:66.129.239.13; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10019020)(4636009)(376002)(136003)(396003)(346002)(39860400002)(2980300002)(189003)(199004)(7696005)(81156014)(8676002)(336012)(117636001)(23726003)(50226002)(53936002)(50466002)(97756001)(76506006)(70586007)(55016002)(9686003)(229853002)(446003)(11346002)(126002)(6246003)(7126003)(8936002)(476003)(6916009)(2906002)(305945005)(4326008)(6266002)(478600001)(486006)(186003)(356004)(26005)(86362001)(81166006)(70206006)(76176011)(4744005)(54906003)(47776003)(5660300002)(97876018)(46406003)(16586007)(316002)(53416004)(107886003); DIR:OUT; SFP:1102; SCL:1; SRVR:BL0PR05MB4692; H:P-EXFEND-EQX-02.jnpr.net; FPR:; SPF:SoftFail; LANG:en; PTR:InfoDomainNonexistent; MX:1; A:1; X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f3fe614f-ff4a-4e92-3156-08d72e231aa8 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600166)(711020)(4605104)(4710121)(4711137)(1401327)(4618075)(2017052603328); SRVR:BL0PR05MB4692; X-MS-TrafficTypeDiagnostic: BL0PR05MB4692: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-Forefront-PRVS: 014617085B X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Message-Info: fIj1RjtuLpmbCmXCFtzuC/d+H8Zlr2oFxNLQ6aNHHwL8KWdDbfh2ew13713fQ2wox71TuqnBbSZPFAmGVbKM7RmkDakbge6ShXTSNwXWgRwF+OQ6XprbT8gwXrLAXRJz3S4tAj93Nls+3y4RHn20Vtgxzt0uK6Nq9cWx7wGdpj27i8HsAPRAR0aIfchYyazhlb4+w70y4ILoUgEfVP3+L3IJdUFpE35uxrWxmQgv6Fk8N8/aB7AMMtmjG9/IBhCSzz/Pql/LVQ6+UHYp7GXqLw4QWJklq2RpIQ4uo6beMiCLc0+niCs6TTdoJrr+UAFUZe9RFuMWkVpw/1fjNIUvxkisFwnM3j648oA9/UaVcU+LhKmpNRkwkc22AK2QnFpvUtP1NgMCQP1tff7GOslRcoe9rjoB3zPSI1mkQz2fqbE= X-OriginatorOrg: juniper.net X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Aug 2019 14:54:19.9765 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f3fe614f-ff4a-4e92-3156-08d72e231aa8 X-MS-Exchange-CrossTenant-Id: bea78b3c-4cdb-4130-854a-1d193232e5f4 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=bea78b3c-4cdb-4130-854a-1d193232e5f4; Ip=[66.129.239.13]; Helo=[P-EXFEND-EQX-02.jnpr.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR05MB4692 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.70,1.0.8 definitions=2019-08-31_05:2019-08-29,2019-08-31 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_spam_notspam policy=outbound_spam score=0 mlxlogscore=650 spamscore=0 mlxscore=0 phishscore=0 lowpriorityscore=0 priorityscore=1501 suspectscore=0 bulkscore=0 malwarescore=0 impostorscore=0 clxscore=1011 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1906280000 definitions=main-1908310173 X-Rspamd-Queue-Id: 46LKBB5rXTz4YlV X-Spamd-Bar: ------ X-Spamd-Result: default: False [-6.95 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.95)[-0.953,0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Aug 2019 14:54:27 -0000 Ed Maste wrote: > > > - The test is difficult to maintain > > This is too. My main complain is that to debug a test case, I must strip > > all atf* to be able to examine it under a debugger. IMO this is a bug in the state of ATF in FreeBSD. The ability to run atf tests on the host in-line with the build is one of the reasons I picked ATF for Junos - but has been discarded in FreeBSD. With atf tests runnable on build host in situ they are not hard to debug there too. I recently added a test suite for my package system (for Junos) and had to use the bmake framework, because FreeBSD made it impossible to use ATF/Kyua > > Yes, this is my biggest complaint about our current test setup. But > this impacts the tests' friendliness to developers, but not their > efficacy in reporting regressions. From owner-freebsd-hackers@freebsd.org Sat Aug 31 17:01:14 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6F662D6B32 for ; Sat, 31 Aug 2019 17:01:14 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46LN0T3D89z3C6n for ; Sat, 31 Aug 2019 17:01:13 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x744.google.com with SMTP id d26so1376077qkk.2 for ; Sat, 31 Aug 2019 10:01:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:cc; bh=TASPvrVnu4iidRxLP6ME7du/hTU8PZXSNp7Ts/viXls=; b=yKMyZw4IFNu8VkgDQHXh+LnnVs4lz4Pk+4TReFK0z1F4WzrxtFBMKgbMEWHJSiyZHk Y1YufKB3bFjFt6V7yLzgrOjbRbSqbmyJpMOwjNlRTajGRP7LbmVhmbhBHIR8TsU/Is4H kVl1bFV9apG4SuxynEqW/JM+7Yqf1mLJK3bXyYdnLgZUagjYibBMDb0NVU9jRThZ5ox4 dS4krOj2XPgF/W/c9t9UF38ujf2LKZ6QqqRWKcMVzr/LLUu2iBLd/3SzE+3mXDH2ms5/ 14b5A+SYDZ8QuR1RkiCNWcTJLxlmMHMMKMKs8IbV5PIHG8aDIppQ4uAL8qlW5qHNmKa8 Mo0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:cc; bh=TASPvrVnu4iidRxLP6ME7du/hTU8PZXSNp7Ts/viXls=; b=YKACLWOZZYhokdLoq9Apvo1xt4cIYRYp1rH/NAaumOnJodUcMZUvvkiqnvaqOcDGAi /C0kOel2TILPsg8n9sLT3r1jPAVCjw2KIA00QAdNVHYkvahwiAVq5ilXUhop+FRSuHAo r+Jh48LS2JRbYN796+kAW5tmnW9/QRvXDlKZslQjyStGCeQ/clDJ/9Jxvir3sjcD7XXb +D/bzn6r+W2rEk2XDOgqHbbJjps2oI/7jhBIDuuZf/HJohQs/PijnFK/yGzLyNLdpbzO R2P4vdZeEWhSpDnnoIMaJgNX3ItSAC9o5LU5Xv9AeNE85b7qZAZ3rH6/f7RFTVtmokft JmKg== X-Gm-Message-State: APjAAAU0bagAcf1xtOntwF+TzAmN/0+lQdiUdgLCg9hSZBTVoT66PE3a +JBIr4DH2JBu4Vf4g5mQx/VHvyo4UUhpIWx96vHcZAs6lds= X-Received: by 2002:a37:30f:: with SMTP id 15mt11586980qkd.240.1567270871089; Sat, 31 Aug 2019 10:01:11 -0700 (PDT) MIME-Version: 1.0 References: <20190829114057.GZ71821@kib.kiev.ua> <20190830065534.GC71821@kib.kiev.ua> <73111.1567263258@kaos.jnpr.net> In-Reply-To: <73111.1567263258@kaos.jnpr.net> From: Warner Losh Date: Sat, 31 Aug 2019 11:00:59 -0600 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy Cc: FreeBSD Hackers , Li-Wen Hsu , fcp@freebsd.org X-Rspamd-Queue-Id: 46LN0T3D89z3C6n X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=yKMyZw4I; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::744) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-1.46 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.979,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-0.98)[-0.978,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-0.95)[-0.951,0]; RCVD_IN_DNSWL_NONE(0.00)[4.4.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MISSING_TO(2.00)[]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-0.56)[ip: (2.42), ipnet: 2607:f8b0::/32(-2.83), asn: 15169(-2.31), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Aug 2019 17:01:14 -0000 After this weeks discussions, I got to thinking about where we were on this topic. I thought I'd spend a few minutes summarizing my impressions so that we don't get bogged down in the details of disagreement, but rather start from where we agree. Here's my take on what that is, but I write this down as the basis for discussion, not to lay down the law. I think there's consensus on the following points: (1) We want CI (2) We want developers to be responsive to breakage in CI (3) At the moment, we have some sub-optimal tools, and when we change those we should evolve the process. (4) Build breakage is something that should be fixed very quickly. (5) People should reach out to the original developer who committed the change when there's breakage ASAP (6) If the original developer can't timely fix the problem, it's OK to either back out the change, or perhaps commit a tiny fix if the original was huge and the fix needed to fix the build is tiny. (7) Breaking tests is a problem, but our tests need to evolve because we have too high a rate of false positives. (8) There's a sliding scale of urgency (Tier 1 build breakage needs to be fixed in a couple of hours, Tier 2 can go a day, Tier 3 can go longer but shouldn't linger). (9) The urgency for a Test regression also is a sliding scale, but given the current state of the tests we need to apply judgement on revert vs fix test vs disable test. (10) Reverts shouldn't be feared, but there's a cost to reverting automatically and there's some desire for developers to have a chance to be in the loop (11) We need to work on the social aspect of reverts to destigmatize them. We might quibble a bit over timelines for the different pieces, but here's what I've noticed the approximate timelines are today (there are exceptions, and I don't have hard data, just my sense from watching the tree, but I think they form a reasonable basis absent better data): * Build system breakage usually is fixed within an hour (eg, I screwed up a Makefile or bsd.foo.mk file somehow). * x86 build breakages are usually fixed in an hour or two (longer over the weekend) * arm and arm64 build breakages are usually fixed within 4-8 hours * Other build breakages are usually fixed within a day or two. * out of tree compiler breakages are fixed on the order of a week. * I have little data on test breakage, but it's my sense most issues are resolved in less than a week. We've been quite reluctant to do reverts to date. They happen, but have usually been initiated by the committer. Li-Wen and others would like to change that to setting firm timelines; start to reset the social aspect of reverts and document the social norms with an eye towards improving things, either within the SVN framework, or the coming git framework. Finally, there's a number of ways we can do this, not limited to the FCP in question. However, nobody has stepped up to drive that initiative. It would be great to have someone sign up to drive revisions of the various internal developer guides to modernize their content to reflect how norms have evolved since the were written (including this topic and the related topic of responsiveness to @freebsd.org email and others...). Warner From owner-freebsd-hackers@freebsd.org Sat Aug 31 18:55:47 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 91142D90AE; Sat, 31 Aug 2019 18:55:47 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-io1-f45.google.com (mail-io1-f45.google.com [209.85.166.45]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46LQXf4txGz3HfJ; Sat, 31 Aug 2019 18:55:46 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-io1-f45.google.com with SMTP id b10so21038519ioj.2; Sat, 31 Aug 2019 11:55:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=3CYs+2ssJzGmNRUbeoLJuS8bh7nRG2o7LCTK6RFQZyM=; b=bOhwMQMEFGsPaEfzGRvMKNtLktHheveFQ76N792lDMksoBV9IpWLooakEBFOUrgQb7 aYjYEdRGzqSI4X0b67kdF+gcFONUNZTB6qfXUzYhJXySakjVJLmSZlqqzRrV6Uzv0cL0 jx+J2QmGEPhkW2PfevSkcTKzNfTGvCJ1hO48VuA6LJ+YGRU1ddM7er0kzvXBjtBx9A8B SAaEiuZ+4ZyGPTgwQ/D8UVzJQjhoJtZzcL4F6iyM+BvW1HvoYDYvRz3hNpD40TmQg8Gx CcnGGsyUZtzkpOM0ouPG6s7IRxc5V/qH1VnA+KIszA9XoP+SGMRO+4Wx80apgIYxMnH7 dJWg== X-Gm-Message-State: APjAAAUDWK5lvMHveCO6PmjH+w8LVZ1bGwXcA4ikSJzHIM0ZgvbLVHGY LwZmEgRVaY8vuRCNHchqu30FoVT3wxoHiPmvZHm/fw== X-Google-Smtp-Source: APXvYqyymnvBEk/A2JM1h+b1SaNbL9HrtJWnBgeyXWCQNI8ov1Xngnd7lCV2WQYBBhB1nuDWwbnFbyU1qrb2yXVLcqM= X-Received: by 2002:a05:6638:73d:: with SMTP id j29mr19655895jad.21.1567277744950; Sat, 31 Aug 2019 11:55:44 -0700 (PDT) MIME-Version: 1.0 References: <20190829114057.GZ71821@kib.kiev.ua> <20190830065534.GC71821@kib.kiev.ua> <73111.1567263258@kaos.jnpr.net> In-Reply-To: From: Ed Maste Date: Sat, 31 Aug 2019 14:55:25 -0400 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Warner Losh Cc: FreeBSD Hackers , Li-Wen Hsu , fcp@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46LQXf4txGz3HfJ X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.45 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-5.25 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.991,0]; RCVD_IN_DNSWL_NONE(0.00)[45.166.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.26)[ip: (-5.59), ipnet: 209.85.128.0/17(-3.34), asn: 15169(-2.31), country: US(-0.05)]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Aug 2019 18:55:47 -0000 On Sat, 31 Aug 2019 at 13:01, Warner Losh wrote: > > (7) Breaking tests is a problem, but our tests need to evolve because we > have too high a rate of false positives. I'm not sure there's consensus on false positives. In particular, Li-Wen has spent an incredible amount of time over more than six months tracking down test failures and submitting PRs. Flaky tests and false positives have generally been addressed by now (by submitting a PR and disabling or xfailing the test). From owner-freebsd-hackers@freebsd.org Sat Aug 31 19:09:53 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1E481D95EE for ; Sat, 31 Aug 2019 19:09:53 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46LQrw2yybz3JJY for ; Sat, 31 Aug 2019 19:09:52 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x72e.google.com with SMTP id u190so9170867qkh.5 for ; Sat, 31 Aug 2019 12:09:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=wuqIQ6VjJLqAmwFu3UeQjakHZH7aJZq7NTKf+mOEtTM=; b=l61dqkLO8zVFZ10+f7CDeWliXqe0iQLbGAnw6OnVhZOaMnHG47YBV1UnQ2Rw5fRPbB OeJFbH0FUR3ePvwjQd3tka9K3rO1r3c1iyzaEmyjhnYRErBT6wv65koPii8UdG9lLQyx 5Z+ZunLunX+lkH1MXVIUET25dDqEUhgcNrn+dPUldyr2EEVzfM4fYuTCB0jqeZc8oifu m28so9IIWE3okj0Rvj+/x6QhrLIS4X3RpVEfRyI6fB7i4QS3GNmK0Ub4REnnUGDE3uRC FGCggOz1c/DpD0+E8aGu+Ohu0+trHP3SPmhr/nfQIjexTf9oS+krBPYWCktekPMNUbR1 a3yA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=wuqIQ6VjJLqAmwFu3UeQjakHZH7aJZq7NTKf+mOEtTM=; b=nVP7Uuhmc0F58nOJEOOCQXMOXgTPMzLREGA9AHhmxgPq54bnruCHPAP7Rw3i4Tvyx9 LMLhbCNzdHmqLGc4KTyivGo18JCbE3lIbMCsKHOl1bZCLxh8T8SZqDYYHDCYp5diKHqc 020Yuu/GYU+GEYKjvrABAhJRv8LvLPvZsYlICrMKMfBSlLdQu2uRuttzobTfrbetszP6 dXA3OTVJxWil47IkhUh17R9ZtYKtMWmohDZfwp0IbBciGzdfqYZD6huBDom6Y5CpKs2e ZGYVVtxRTIAgFbnLEptdqZFJeEJUFKHIlHxuzP0ZnWzjrIyZCRaxMfo4qKqVxqkgqEm3 WODg== X-Gm-Message-State: APjAAAXsj6Wq2SpY1lxtTPnKnjJ3fh6M2tzUKyLnEqLXt2cV12z6f7Y3 e9Ri3XnXU6H+R1IA74QIt4CylyeoTC6zk8Jt6sWdvg== X-Google-Smtp-Source: APXvYqy0i3b2EuSlq92DA4ISuvDNoUY1nkICcebV1jzGWzapPJyw3Sm6pVFG67D/oSb/ECBBCLiJRNBkDEDBfRVGtvI= X-Received: by 2002:a37:30f:: with SMTP id 15mr12747587qkd.240.1567278590149; Sat, 31 Aug 2019 12:09:50 -0700 (PDT) MIME-Version: 1.0 References: <20190829114057.GZ71821@kib.kiev.ua> <20190830065534.GC71821@kib.kiev.ua> <73111.1567263258@kaos.jnpr.net> In-Reply-To: From: Warner Losh Date: Sat, 31 Aug 2019 13:09:38 -0600 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Ed Maste Cc: FreeBSD Hackers , Li-Wen Hsu , fcp@freebsd.org X-Rspamd-Queue-Id: 46LQrw2yybz3JJY X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=l61dqkLO; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::72e) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-5.90 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-0.996,0]; RCVD_IN_DNSWL_NONE(0.00)[e.2.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.91)[ip: (-9.34), ipnet: 2607:f8b0::/32(-2.83), asn: 15169(-2.31), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Aug 2019 19:09:53 -0000 On Sat, Aug 31, 2019, 12:55 PM Ed Maste wrote: > On Sat, 31 Aug 2019 at 13:01, Warner Losh wrote: > > > > (7) Breaking tests is a problem, but our tests need to evolve because we > > have too high a rate of false positives. > > I'm not sure there's consensus on false positives. In particular, > Li-Wen has spent an incredible amount of time over more than six > months tracking down test failures and submitting PRs. Flaky tests and > false positives have generally been addressed by now (by submitting a > PR and disabling or xfailing the test). > Fair enough. Do we have recent data that can distinguish between false positives or bad perception? Warner > From owner-freebsd-hackers@freebsd.org Sat Aug 31 20:34:03 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2DEE8DB354; Sat, 31 Aug 2019 20:34:03 +0000 (UTC) (envelope-from shreyankfbsd@gmail.com) Received: from mail-yb1-xb36.google.com (mail-yb1-xb36.google.com [IPv6:2607:f8b0:4864:20::b36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46LSk21KCwz3N0K; Sat, 31 Aug 2019 20:34:01 +0000 (UTC) (envelope-from shreyankfbsd@gmail.com) Received: by mail-yb1-xb36.google.com with SMTP id t15so3698562ybg.7; Sat, 31 Aug 2019 13:34:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=d1frEj6sgThf89efHLok+CyF24QjJzhPZrryN/OCMTs=; b=k1FhwzuNAwUnkFPLoLubXymbNOfczAUlNUTDV0i3Y6vwCxJI/GNgWiIBT05B7Bhud9 F9dpG5kkdrbBvsU9U7VI0+wyjjYA6cmzvLWd8qECXkyeEjsO2IEIFRRlde6y1u/vVuzt CT8BVKT1nuv+InFq0tgQ3yqTX92JTfIGVZlRicbnu7lwo7Y/u0zye9UkKwPvCXYQsQn9 XlB8+Z7YqC6etLr3JV/bixFK1xeCOaXAT9NnRbUOGVmOev0v6GBf9IQEBPaiX3/jn8pK wY7Jtg1KIAb6Qsqfj7UbLjKL4PqEWXA/iEjuSGqSzNbJrlTRPve3FLqRU16xHzL4r0oy ygOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=d1frEj6sgThf89efHLok+CyF24QjJzhPZrryN/OCMTs=; b=g4zTOmIolkSqztBf7I/P4JEuln11GhvRF4p44ezDtHajxsdZkfw5m4ctt1ATXtVqmt BLAoq63ghkhR/JJmCaqqm7dHqHmVvA/74GHFwvduyltZchIrZntp1bPYVeFvrc7D2OZr zKbh4rMVs/2FUnhWPE4g/rcafrtCq6oARJZXaleaPmiIDnBE1b13Dutn6FU9nHCZAoKe 7VZC8Nrde/3Ieic6yg0YEFBKsxET4VvDdC5uNCsR4vR8040FJWT2thqHQwbt7lZgMCaz +y/JInkixo0bAI7r24MNN+adGznS1tdVo4QPsIqVs0mbolHV/zkOEXQC9gUS3aj/bD4g 7WYg== X-Gm-Message-State: APjAAAWER6BKCZvdj+HWl0wt0AKCk1yitc84aXD3+p7o8yCna3e+1pLg 0WGt2CKw9JbRv1MAYQA80DQHvR4cDJ2G8dqPT0Y9Fdw= X-Google-Smtp-Source: APXvYqwL6IFML+zrjdQK7IW/RZE/02MGaVUY6QvWbPQ82UI2M9x1+vtoEvNL6jh1LcwaG+pUDAaOEKhWjnau6GIpZkc= X-Received: by 2002:a25:5ca:: with SMTP id 193mr16383919ybf.388.1567283640294; Sat, 31 Aug 2019 13:34:00 -0700 (PDT) MIME-Version: 1.0 From: shreyank amartya Date: Sun, 1 Sep 2019 02:03:48 +0530 Message-ID: Subject: hwpmc NMI/cpuxx ... going to debugger To: freebsd-drivers@freebsd.org Cc: freebsd-hackers@freebsd.org, avg@freebsd.org, mmacy@freebsd.org X-Rspamd-Queue-Id: 46LSk21KCwz3N0K X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=k1FhwzuN; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of shreyankfbsd@gmail.com designates 2607:f8b0:4864:20::b36 as permitted sender) smtp.mailfrom=shreyankfbsd@gmail.com X-Spamd-Result: default: False [-3.99 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; IP_SCORE(0.00)[ip: (-9.45), ipnet: 2607:f8b0::/32(-2.83), asn: 15169(-2.31), country: US(-0.05)]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[6.3.b.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.99)[-0.994,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Aug 2019 20:34:03 -0000 Hi, I'm trying to profile a java application using hwpmc. While running the profiler I run into these errors: NMI ISA 30, EISA ff NMI/cpu123 ... going to debugger NMI ISA 30, EISA ff NMI/cpu58 ... going to debugger NMI ISA 20, EISA ff NMI/cpu59 ... going to debugger Due to this, pmclog contains very few samples and sometimes none at all. I enabled hwpmc kernel traces and with that it seems that cpu which is unable to catch the pmc interrupt hits true for this condition in amd_intr(hwpmc_amd.c): if ((pm = pac->pc_amdpmcs[i].phw_pmc) == NULL || !PMC_IS_SAMPLING_MODE(PMC_TO_MODE(pm))) { isnull++; continue; } if (!AMD_PMC_HAS_OVERFLOWED(i)) { ovrflw++; continue; } These are some logs Aug 31 20:01:11 amd kernel: cpu245 MDP:INT:1: cpu=245 tf=0xfffffe00015cbf30 um=0 Aug 31 20:01:11 amd kernel: cpu245 MDP:INT:2: retval=1 isnull=0 ovrflw=0 Aug 31 20:01:11 amd kernel: cpu245 MDP:INT:1: cpu=245 tf=0xfffffe00015cbf30 um=1 Aug 31 20:01:11 amd kernel: cpu245 MDP:INT:2: retval=1 isnull=0 ovrflw=0 Aug 31 20:01:11 amd kernel: cpu245 MDP:INT:1: cpu=245 tf=0xfffffe00015cbf30 um=0 Aug 31 20:01:11 amd kernel: cpu245 MDP:INT:2: retval=0 isnull=16 ovrflw=0 Aug 31 20:01:11 amd kernel: NMI ISA 3c, EISA ff Aug 31 20:01:11 amd kernel: NMI/cpu245 ... going to debugger Aug 31 20:01:11 amd kernel: cpu236 MDP:INT:1: cpu=236 tf=0xfffffe0001595f30 um=0 Aug 31 20:01:11 amd kernel: cpu236 MDP:INT:2: retval=1 isnull=0 ovrflw=0 Aug 31 20:01:11 amd kernel: cpu252 MDP:INT:1: cpu=252 tf=0xfffffe00015f5f30 um=0 Aug 31 20:01:11 amd kernel: cpu252 MDP:INT:2: retval=1 isnull=0 ovrflw=0 Other observations: If I enable all the md=* flags, I do not see this issue. When i run the profiler for a C++ application, I do not encounter this problem. There is no panic observed. Any idea on what could be causing this? or pointers on how to debug this further? Thanks Shreyank