From owner-freebsd-questions@freebsd.org Wed Jun 16 23:12:59 2021 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0865964716F for ; Wed, 16 Jun 2021 23:12:59 +0000 (UTC) (envelope-from kh@panix.com) Received: from mailbackend.panix.com (mailbackend.panix.com [166.84.1.89]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4G51F62cB0z4YnH for ; Wed, 16 Jun 2021 23:12:58 +0000 (UTC) (envelope-from kh@panix.com) Received: from rain.home (pool-96-230-243-2.bstnma.fios.verizon.net [96.230.243.2]) by mailbackend.panix.com (Postfix) with ESMTPSA id 4G51F53Hq6z4925 for ; Wed, 16 Jun 2021 19:12:57 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=panix.com; s=panix; t=1623885177; bh=3gl4kE5u+dpnCPphl/8n0eEAAGSWwdwVvu7e5QDgFqo=; h=Subject:To:References:From:Date:In-Reply-To; b=Hj28Ck8Tfl0RxZrSqKSu00UkLrk5DEyf1oR2V/uM9GNdiuobJrxRiQXstOR2X/KnL laQ16H24H9rhI9wsMNNjV2fkbGfvVuD6Mit7KoGGSEhduw+cCyhuhrH7TsNU65vXou I45E4DMQfeUVE/kqnoRKtkWf5OlQlfyChn8pz890= Subject: Re: Is a successful call to write(2) atomic? To: freebsd-questions@freebsd.org References: <31698.1623883195@segfault.tristatelogic.com> From: Kurt Hackenberg Message-ID: Date: Wed, 16 Jun 2021 19:12:54 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: <31698.1623883195@segfault.tristatelogic.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4G51F62cB0z4YnH X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=panix.com header.s=panix header.b=Hj28Ck8T; dmarc=none; spf=pass (mx1.freebsd.org: domain of kh@panix.com designates 166.84.1.89 as permitted sender) smtp.mailfrom=kh@panix.com X-Spamd-Result: default: False [-2.20 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[166.84.1.89:from]; R_SPF_ALLOW(-0.20)[+ip4:166.84.0.0/16]; TO_DN_NONE(0.00)[]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; RCVD_IN_DNSWL_MED(-0.20)[166.84.1.89:from]; DKIM_TRACE(0.00)[panix.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:2033, ipnet:166.84.0.0/16, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[96.230.243.2:received]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[panix.com:s=panix]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; DMARC_NA(0.00)[panix.com]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[panix.com:dkim]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-questions] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jun 2021 23:12:59 -0000 On 2021/06/16 18:39, Ronald F. Guilmette wrote: > The only reason I mention that is because I've just tried changing my > code so that the FD (#1) gets O_APPEND set on it at startup time and > then the calls to write() are wrapped in calls to flock() just as > John Levine and others suggested. Result? Output is still garbled. John Levine's sample code opened and closed the file around each write. Here is that code: fd = open("somefile", O_CREAT|O_WRONLY|O_APPEND); /* put some stuff in buf[] */ flock(fd, LOCK_EX); write(fd, buf, strlen(buf)): /* O_APPEND ensures it's added at the end */ flock(fd, LOCK_UN); Since you only open the file once, you should seek to end of file before each write, as somebody else pointed out. You should do that seek, and the write, while holding the file lock. That is: lock file seek to EOF write unlock file Does your code do that? The reason, of course, is that some other process could move end of file out from under you while you do not hold the file lock. > The one really odd thing about all of this is that I already have, and > have had, for a long time now, *many* programs that I've built and that > I have been using, also for a long time, that follow this same general > model, i.e. a parent a a lot of "worker bee" child processes where each > of the children, after completing a work item, does itself write a single > line of output, and *those* programs have all seemed to work flawlessly > (i.e. no output garbling) over a long period of time.... HOWEVER they > are all using the stdio functions to write to stdout, rather than > calling write() directly to write to FD #1. Those C library functions buffer the I/O operations within the user-mode process. The system calls don't do that. That C library buffering could change the internal details of concurrent access. > OK, so I tried swapping out calls to write() to call to fwrite()... > > Result: Some output lines still garbled. No surprise. You should lock the file, seek to end, write, unlock.