From owner-freebsd-current Wed Mar 27 22:25:53 2002 Delivered-To: freebsd-current@freebsd.org Received: from horsey.gshapiro.net (horsey.gshapiro.net [209.220.147.178]) by hub.freebsd.org (Postfix) with ESMTP id 2C9E737B400; Wed, 27 Mar 2002 22:25:29 -0800 (PST) Received: from horsey.gshapiro.net (gshapiro@localhost [IPv6:::1]) by horsey.gshapiro.net (8.12.3.Beta2/8.12.3.Beta2) with ESMTP id g2S6PSGd036797 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Wed, 27 Mar 2002 22:25:28 -0800 (PST) Received: (from gshapiro@localhost) by horsey.gshapiro.net (8.12.3.Beta2/8.12.3.Beta2/Submit) id g2S6PSbJ036794; Wed, 27 Mar 2002 22:25:28 -0800 (PST) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15522.46936.107162.91222@horsey.gshapiro.net> Date: Wed, 27 Mar 2002 22:25:28 -0800 From: Gregory Neil Shapiro To: freebsd-stable@freebsd.org, freebsd-current@freebsd.org Subject: The sendmail discussion... X-Mailer: VM 7.00 under 21.1 (patch 14) "Cuyahoga Valley" XEmacs Lucid Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I've been purposefully trying to avoid getting involved with the entire "should sendmail be in the base OS" debate as my input would obviously be biased. However, avoiding a response has become more and more difficult as I've seen unanswered questions, misinformation, and as of late, people either posting what they believe my views to be or asking that I post. As you read this, keep in mind these are *my own personal opinions* and apply all of the standard disclaimers that implies (e.g., some of my facts may be wrong). It's ok if you don't agree with them. I also apologize for the length, but people wanted to hear my views. Think of it this way, it is in reply to 71 posts in the current thread. Feel free to skip sections if you want. The last couple of sections may be of interest to those looking for a solution. So here goes... ---------------------------------------------------------------------------- My place in all of this. I've been involved in supporting sendmail since 1996, once of the primary developers since 1997, working for Sendmail, Inc. since 1998, and finally working on the FreeBSD sendmail distribution since 2000. Someone wrote, "Greg Shapiro--who's FreeBSD work may even been supported by Sendmail, Inc." For the record, Sendmail, Inc. does not pay me to keep sendmail up to date in FreeBSD. They pay me to work on the actual sendmail source code and other commercial products. The work on FreeBSD is purely voluntary on my part. That same person wrote, "Greg ensures that 'it ain't broke, so don't fix it.'" Thanks. :) Another statement I would like to correct was: 3) We'd lose the contribution of Greg if sendmail was moved out of the core system. (Could this possibly be true?? I assume that Greg would eventually become involved in the discourse if it looked like something would actually happen, and his veto would definitely shut down the possibility of doing any of this. Some people are just That Important.) I think people overestimate my role in FreeBSD. I'm only one committer. I don't dictate what is moved in or out of the core system. Ultimately, the FreeBSD community, through the mailing lists, PR and patch submissions, and volunteering, drive the future of FreeBSD. I don't have veto power, I am just "Not That Important". I think core@freebsd.org is the only group who is "Just That Important". I do however worry that sendmail will be moved out of the core system. I've tried to answer the technical reasons below. But on a personal note, sendmail's existence in FreeBSD is what allowed me to become a committer and I would hate to lose the ability to contribute if it were removed. ---------------------------------------------------------------------------- Why is sendmail in FreeBSD in the first place? The reasons are clearly historical. Someone asked, why "non-BSD packages" are in the distribution. sendmail started as part of the CSRG Berkeley Software Distributions. It wasn't added to BSD or added to FreeBSD, it was just there. ---------------------------------------------------------------------------- Why is sendmail still in FreeBSD? In FreeBSD's case it is still sendmail for what I believe to be three reasons: 1. It's traditionally been sendmail. Don't underestimate the importance of tradition. No, it doesn't necessarily make it right but it does maintain continuity. 2. A large portion of the FreeBSD community use sendmail. Switching it out on them now would cause a large amount of hassles. Users who don't use sendmail have already figured out how to deal with the changes necessary. Switching it out on the others would cause more damage than good. 3. It's my fault. I've heard rumors that before I became a committer, sendmail may have been removed from FreeBSD because it wasn't being actively maintained. I don't know if these rumors are true, but if you are looking for someone to blame, look no further. Someone stated that the "underlying reason that sendmail and bind remain in src-contrib is that the maintainers are developing/maintaining at a rate that would make generating port patches and doing port versions prohibitively time consuming." While it is true that packages like sendmail and BIND are under active development, they are not actively developed in the FreeBSD repository. Port versions actually come out faster than base OS versions. Released versions are however imported into the FreeBSD repository. However, you are correct in that the infrastructure surrounding sendmail (e.g., src/etc/mail and the buildworld components) are actively maintained. That probably has kept it in. Someone asked why "large, complex" packages are part of the system. Personally, I don't think things should be pulled out because of their size or complexity. There are many parts of the FreeBSD system that fall under this category, not just the favorite targets of sendmail and BIND. However, I've really worked hard at mitigating the complexities involved. I've tried to make it much easier to configure and maintain sendmail in FreeBSD compared to it's state a few years ago. Most of this work has been in /etc/mail/Makefile and the build infrastructure (for those using buildworld/installworld to upgrade). ---------------------------------------------------------------------------- Ok, history aside, why is sendmail still in FreeBSD? Every commercial and open source UNIX operating system ships with an MTA, most of them sendmail. I do not think it would a good idea to ship a UNIX or UNIX like operating system without an MTA. However, no, it doesn't have to be sendmail. I completely, 100% agree that users should be given a choice. See "What are the problems in the current setup?" and "Where do I think things should go?" below for more of my thoughts on this. ---------------------------------------------------------------------------- Why is there a sendmail port? The sendmail port(s) serves a different role than the code in the base operating system: 1. The ports usually have the latest version of sendmail available on the day of the sendmail release. On the other hand, I don't import every release on the the day of the release. Call me a wimp but I prefer to let releases shake out a little before forcing them on everyone. Witness the time delay in getting sendmail 8.12 into the tree. I also wouldn't be honest if I didn't say that I sometimes don't have time to do it immediately. 2. The sendmail built with the OS only includes support for items that come with the base OS. The port allows users to build sendmail with features not available in the base OS such as SASL and LDAP. 3. The ports have the ability to track beta versions of sendmail. 4. The ports give users a way to modify the way sendmail is built without requiring them to keep /usr/src up to date and build world. While many of us are quite comfortable doing a make buildworld and make installworld (on a running multiuser system no less), other users wouldn't even know where to begin. 5. The ports can enable experimental features I am not willing to turn on in the base OS. For example, the port offered milter support during the 8.11 release. 6. The ports can be upgraded to new versions while the operating system isn't upgraded. For example, some users out there are still running FreeBSD 3.X (or even 2.X) and can use the ports to get sendmail 8.12. Dirk Meyer does a great job at actively maintaining the port and I personally appreciate his hard work. ---------------------------------------------------------------------------- Why did you change the infrastructure in the first place? Much of this debate started when I started MFC'ing 8.12. Unfortunately, the questions didn't come up when I posted the -CURRENT patch for review, when I committed 8.12 to -CURRENT, when I waited weeks for comments before MFC'ing, when I posted a patch to -STABLE for review. People waited until *after* I started committing to -STABLE. But, no, I'm not bitter about that. :) (Note: humor, don't flame me) I had to change /etc/rc so that if sendmail_enable was set to NO, a submit-only daemon was started. (Due to a bug in the new /etc/rc, an extra queue running daemon was also started. That is now fixed in -CURRENT and will be MFC'ed in 1 week unless something comes up.) First a little bit of technical info on why a new daemon is needed. sendmail 8.12 is no longer set-user-ID (and there was much rejoicing). Command line submissions are relayed to an MTA (which usually runs as root, but doesn't have to) for processing. This makes the command line invoked sendmail act as a pure relay translating the command line and standard input into an SMTP stream to talk to an MTA. See /etc/mail/README for more information. Many were understandably upset that setting sendmail_enable to NO didn't prevent any sendmail daemons from starting. I thought long and hard about how to cause the least amount of damage. I honestly did (and still do) believe my solution provides a working system to most of the user community. Users who were using sendmail and had sendmail_enable set to NO would still have a working sendmail installation after an upgrade as a daemon listening on localhost only would be started to accept command line submissions. Without that daemon, mail would be queued in the submission mail queue for ever without any indication. An additional queue running daemon was also added for the submission mail queue in case the MTA wasn't answering at the time mail was submitted on the command line. Again, this is needed to avoid queuing mail forever. More technical information is also available in /usr/src/contrib/sendmail/src/SECURITY. I've now added support for sendmail_enable=NONE to prevent *any* sendmail daemons from being started. This will be MFC'ed in 1 week unless something comes up. Until the MFC, -STABLE users can turn off all of the sendmail startup code with: sendmail_enable="NO" sendmail_outbound_enable="NO" sendmail_msp_queue_enable="NO" sendmail_submit_enable="NO" I was actually surprised to learn some people were starting other MTAs using sendmail_enable="YES". I was under the (false?) impression that all port-installed MTAs had their own /usr/local/etc/rc.d/ startup scripts. The infrastructure changes in place are necessary to support sendmail 8.12 as a non-set-user-ID binary. In my opinion, the inconvenience added by requiring alternative MTAs to change sendmail_enable from NO to NONE is minor compared with getting rid of a set-user-ID root binary in the OS distribution. I'm sorry for any inconvenience I have caused. I will work with Warner and Bruce to get a note into both /usr/src/UPDATING and the release notes. For those who have suggested changing the name from "sendmail_enable" to "mta_enable", I don't think that solves anything. The startup routines in /etc/rc and the settings for sendmail_*_flags in /etc/defaults/rc.conf are truly sendmail oriented. I've never envisioned "sendmail_enable" to be non-MTA specific. ---------------------------------------------------------------------------- What are the "problems" in the current setup? 1. The build time configuration in /etc/make.conf has no influence on the startup time or run-time configuration. I personally feel this is correct and not a problem or bug. /etc/make.conf is for building, not for configuring services. When it comes to building, I've tried to offer as many sendmail build knobs as I could to satisfy everyone's needs. 2. People installing from release media get sendmail installed. Yes, this is a problem. People should be able to chose the components they want and the current selection of 'bin', 'doc', 'man' is too coarse grained. Separating FreeBSD into packages is a good idea but it should be done across the board instead of singling out the MTA. People should be able to pick and choose what they want to install. Ideally, everything except the core OS essentials should be optional. This includes items such as OpenSSH, BIND, sendmail, gcc, perl, cvs, X11, ISDN, UUCP, groff, NIS, lpr, amd, etc. However, realistically, this would require a lot of work and FreeBSD is a volunteer effort. Also, in some cases, alternatives or simple replacements will be needed. For example, it would be great for users to be offered other MTAs such as postfix or exim. For those who don't want any MTA, a specialized SMTP client (to take command line submissions and turn them into an SMTP submission to a real MTA) could be written but great care must be taken -- it's not as simple as it sounds (think error conditions, queuing, etc. and keep in mind the tool would need to handle submissions from automated processes such as cron which can't resend later if there is a problem). The pre-formatted man pages would need to be installed if groff wasn't added. Any perl scripts in the base OS would need to be converted to shell scripts or C code (not including the perl stuff to buildworld or buildkernel -- you can require that people who expect to build things have gcc and perl installed). 3. People installing from installworld get sendmail installed. As far as I know, there are sufficient knobs in place to prevents this (NO_SENDMAIL, NO_MAILWRAPPER or using mailwrapper). 4. People installing from ports get multiple versions installed There are a couple of things that can mitigate this problem. The port can give users instructions (or a script) on how to remove the base installed version. If the release media problem above were solved, it would help in this situation as well. Those building/upgrading from source already have the tools to prevent the base OS install. 5. The startup scripts get in the way of alternative MTAs The startup scripts provide knobs to disable the startup of virtually every daemon in the base OS. Yes, occasionally these knobs will change, and yes, occasionally new ones will be added. That is unavoidable in a system that is maintained. The alternative is to never upgrade anything in the OS. Those who prefer that should use one of the release branches (e.g., RELENG_4_4). As a side note, I think it is perfectly acceptable to have this happen in -STABLE. Only releases are frozen points in time, adding new things to -STABLE which change behaviors are acceptable in my opinion. That is why it is called -STABLE instead of -FROZEN or -DEAD. (I'll probably get some flack for this one). In my opinion, the items in /etc/rc and /etc/defaults/rc.conf are for controlling the daemons included with the OS. As the daemons in the OS change, so do the startup routines. Also, I don't expect the code for sendmail_enable to properly work for starting exim. Hence the 'sendmail' in the name. I really would like to see work in the /etc/rc.d project resume. I think this would solve a lot of the recent problems that have been brought up. ---------------------------------------------------------------------------- Where do I think things should go? If the project (and/or I) had unlimited time and resources: 1. Convert FreeBSD into a bunch of packages and let binary installations with sysinstall (or some future installer) pick and chose or pick from a short list of "standard" systems. Offer alternatives where available. 2. Finish the /etc/rc.d/ work so it mirrors the /usr/local/etc/rc.d/ system the ports already use. That way a buildworld with NO_SENDMAIL=yes in /etc/make.conf or not installing the sendmail "package" in the future sysinstall (see previous item) would prevent an /etc/rc.d/sendmail from being installed. I plan on continuing to improve the FreeBSD infrastructure for sendmail and will continue trying to be sensitive to the needs of non-sendmail users. I welcome feedback and I try to be quite reasonable. ---------------------------------------------------------------------------- On a personal note... I'd like to quote two statements that were made in the thread and respond to them since they encompass my pet peeves: - "I hate sendmail with a passion." sendmail is just a piece of software. I gave up being passionate about that sort of thing a long time ago and have a lot less stress in my life now. I can understand someone disliking a piece of software and choosing not to use it, but hating it with a passion? - "Darn sendmail trash.." I truly wish people could have a technical debate without mudslinging. You aren't insulting the bits on the disk with statements like this, you are really insulting the authors of that software. Let's try to be civil. Please keep the discussion technical and not resort to personal attacks and mudslinging. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message