From owner-freebsd-questions@freebsd.org Tue Jun 15 06:55:29 2021 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1B647641808 for ; Tue, 15 Jun 2021 06:55:29 +0000 (UTC) (envelope-from pprocacci@gmail.com) Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4G3zbh1CXdz4fXX for ; Tue, 15 Jun 2021 06:55:27 +0000 (UTC) (envelope-from pprocacci@gmail.com) Received: by mail-pj1-x1031.google.com with SMTP id 13-20020a17090a08cdb029016eed209ca4so891889pjn.1 for ; Mon, 14 Jun 2021 23:55:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=tqFvgS1KTHwY1TcriJqQIpk9LbfgB9z3EmNyt6xYjc0=; b=VnYI6uki9sEiYxW2AKQer5k+Y2T5RFxhzad6ZbFJYlYCq4UpCB69U9VOqcaVrbKstJ omLLbEqmnvywI+NzUxFXY/FaSh5PPw6kvRDlZnKoECEf+ZmkN70bEKHsnpknUXIKmllv ShTUP5keKEWRu7ZIx9WczDUgohJMx99DB4+eWqAd2ExmQyMraNvJe8NLMuwQRNUeq7qJ RoqcJpsil5nPcYeSUJURAxVnRm9C5eM5PHGktZS1CsdUjbWfWFFuQHrP2CrgJ9/i6Rdk NzMZ02uFGpiCbn9I7N0Yzzor78g2KGKPjXjw2mvDNEif7FQGwYPGk0l5h0HLDmIcafDd NjoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=tqFvgS1KTHwY1TcriJqQIpk9LbfgB9z3EmNyt6xYjc0=; b=rlvMr3Hi6fat7m7LLq8UQkhrVupnxb86vPkLCaBr3nGqmV0VM8T6OKYfhoq8M75hcL Ju4tGcyYPLHzD24BUX6DL/gjDX45ncvRFlTrHyJ6FpNxg8xi6ryZAROtZtrvJr6iBPpa dqwnR6UWYsL59MIfHmdlxykOZBCOVh+XMaO5vlesWwB3pZo4a778ELHmJrlpmUNVhqCr 3XrE9shNTFFoi+IWxmlsBkq1ePRRZJcxC4mIhczEx32/boYQN+5JsHG94L+TWaJYp6lW D4pygndC1iRgJ44lIpuQ8rsfhaLGkd7y4KjkkJcpHNPA/czuYm721hCZDdQ8nfDdJ8yt 7PzA== X-Gm-Message-State: AOAM533oX/XIZFASZGb+Gy5OwGPIrYqGIbBxE6qdBuoNZHWphzI3rFe+ X9CZ50YHW6qTUdjzvqIHJ2RAzTLeYDfzrxMHG0YgIUs42wMr X-Google-Smtp-Source: ABdhPJxqIg4Ckh37vqQc+EdA4cPvoIk4zG221Toe4/FsHnyoHPOO7mo0vGbSMUv7MMP5fxt8Idx758yrqtw7ibh7sd4= X-Received: by 2002:a17:90a:694d:: with SMTP id j13mr23642981pjm.99.1623740126274; Mon, 14 Jun 2021 23:55:26 -0700 (PDT) MIME-Version: 1.0 References: <22223.1623737465@segfault.tristatelogic.com> In-Reply-To: <22223.1623737465@segfault.tristatelogic.com> From: Paul Procacci Date: Tue, 15 Jun 2021 02:55:14 -0400 Message-ID: Subject: Re: Is a successful call to write(2) atomic? To: "Ronald F. Guilmette" Cc: FreeBSD Questions X-Rspamd-Queue-Id: 4G3zbh1CXdz4fXX X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=VnYI6uki; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of pprocacci@gmail.com designates 2607:f8b0:4864:20::1031 as permitted sender) smtp.mailfrom=pprocacci@gmail.com X-Spamd-Result: default: False [-1.00 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::1031:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(1.00)[0.998]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::1031:from:127.0.2.255]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::1031:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-questions] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jun 2021 06:55:29 -0000 On Tue, Jun 15, 2021 at 2:11 AM Ronald F. Guilmette wrote: > > Assume that a given program, "example", forks a child process (or perhaps > many such) and then each of these processes tries, in a manner that is > entirely non-sychronized with any of the others, to write blocks of data > using calls to write(2) to file descriptor 1. > > Assume further that "example" is run from the command line as follows: > > example > ordinary-disk-file > > Lastly asume that the return value for all calls to write() is diligently > checked to insure that it is never either -1 or a short write count, > and that if it ever, is, "example" will exit with a non-zero exit status > (and assume also that otherwise it will exit with a zero exit status). > > Question: Assuming that "example" runs and exits with a zero status (thus > indicating that all calls to write() completed successfully and none > returned > a short write count) then might it ever be the case that some single chunk > of data that was successfully written to "ordinary-disk-file" by a single > call to write() might be split into two or more parts in the output file, > with chunks of data that were written by other processes in the same family > possibly being interleaved between parts of that one chunk in the output > file? > > I had a long response, but decided to shorten it....I hope you don't mind. The only time atomicity is guaranteed is for pipes when the write(2) size fits in PIPE_BUF (512 bytes on my version of FreeBSD). Otherwise, it depends is really the best answer as it depends on the file system. Sometimes filesystems like ext4 require O_DIRECT to be set, in order to avoid the garbled text as you've mentioned, and even then it's limited to the number of bytes written. A suggestion; the way I've done something similar in the past..... Parent Process - Orchestrator - creates pipes handing one end of the pipe to each child. Child Process - do'ers - write whatever it intends on writing to the pipe the parent is waiting for data on. Parent process then grabs the data the child wrote, and in turn writes it to stdout (or whatever file descriptor you intend). This method, a bit more involved, ensures one writer to a file descriptor, is portable, and you can rest easy about it working on whatever latest and greatest file system someone decides to make. That's always been my understanding anyways, and I am open to being corrected. ~Paul -- __________________ :(){ :|:& };: