From nobody Sun Jul 9 12:44:04 2023 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QzRdx0mcqz4m4Ck for ; Sun, 9 Jul 2023 12:44:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QzRdw3dCtz41Tg for ; Sun, 9 Jul 2023 12:44:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1688906644; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Pn2eQc9h5Ao7lN6S6aoCa8jnkBb6+BA89p2c0cy8oDk=; b=wLEnXO2IkX3BhTlNhCVb2kZMlnC7sG7N9UabrD75rVCStd74Di3e+LJI3VJuBXt+nDXKlJ xmlmYfnI8KQrfbPxk46z4xHF+xtd5zG1TlxvFrKyYVtVaK+tm3Nepxg6XZl/aSBOZFXNvC BE0f6i/worCh2CaPbjOTDKauo7MHVd4pAPtONoom9s16/FQxUEmMBex2EhSc1Yjr2ip3Fh TLd38QIjlvIwqRUYke1TKs747kmaVTRtbzdmfvCfJug1/5g3icjiVUfesopprkyOFLzjqp vzST/3i5LjJjJKxGHxuq+Jt23TLzTwVtQ3ypAV3Rr9L1Tfnfxw9LcZmn8Elvag== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1688906644; a=rsa-sha256; cv=none; b=fbM5OYV0NMGClRhRdSZfQ9bzmsexhbg3jjnThleCTFQayncuLYkBKiy7D7V6Vqts8m5+CI Ls7B6px2gXwFYkfWouZRgKXbG7tmmsm1VlfOHn8EHQZffEhltO9trOJRLkUnS3koY2Wet7 V7FTP4mWPmPPkKk4M5t/hmScEHNc5uEuxSBGS5IbsXsJbcvDd/uWHePIUvPt296ZmZwRz8 MiBxZ3pId48+4y0CKyg0eRvKC0xpt7AT/WFLzC0h0cMcxQduR7I0F8COIVglbRIbGg7cq9 aJx46DJisMS+Y3d7zkrQ+ehbDfMH1gRrM8OlZfNoShGwNmgD+AsdRdWH52xy/Q== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4QzRdw2hbqz13lT for ; Sun, 9 Jul 2023 12:44:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 369Ci4gg055148 for ; Sun, 9 Jul 2023 12:44:04 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 369Ci43b055147 for bugs@FreeBSD.org; Sun, 9 Jul 2023 12:44:04 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 272434] du and disk operations (like mv,cp,shutil.move/rmtree,os.rename) combination issues Date: Sun, 09 Jul 2023 12:44:04 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: meetshamsher@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D272434 Bug ID: 272434 Summary: du and disk operations (like mv,cp,shutil.move/rmtree,os.rename) combination issues Product: Base System Version: 13.2-RELEASE Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: meetshamsher@gmail.com Hi, We were performing file system operations using du,cp,mv,shutil.move/rmtree= and os.rename. Observed one of the directory is going for toss in 13.1 and 13.2 (also checked in iso of 13.0,12.4 same issues observed). Yet to verify in current. rtestuser@freeBSD13_2:~ $ uname -a FreeBSD freeBSD13_2.cs39 13.2-RELEASE FreeBSD 13.2-RELEASE releng/13.2-n254617-525ecfdad597 GENERIC amd64 rtestuser@freeBSD13_2:~ $ cd /data/infra/ rtestuser@freeBSD13_2:/data/infra $ ps -aux | grep python root 726 0.0 9.1 23100 8768 v0 I 14:34 14:51.95 python move-move-delete.py (python3.9) root 3332 0.0 7.3 18800 7028 v0 I 14:35 0:00.03 python du.py (python3.9) root 23350 0.0 7.3 18800 7028 v0 I 15:07 0:00.02 python du.py (python3.9) rtestuser 48818 0.0 2.1 12812 1984 0 S+ 15:53 0:00.00 grep python rtestuser@freeBSD13_2:/data/infra $ ls createfiles.py dir1 dir4_ren du.= py=20=20 move-move-delete.py rtestuser@freeBSD13_2:/data/infra $ cat move-move-delete.py=20 import os import shutil import time path2 =3D "/data/infra/dir2_ren" path3 =3D "/data/infra/dir3_ren" path4 =3D "/data/infra/dir4_ren" while 1: os.rename("/data/infra/dir1/","/data/infra/dir2_ren/") os.system("ls -ir /data/infra/") print("rename done") os.mkdir(path3) files =3D [f for f in os.listdir("/data/infra/dir2_ren/")] for file in files: old =3D "/data/infra/dir2_ren/" + file ipath =3D "/data/infra/dir2_ren/" + file + "/" files1 =3D [af for af in os.listdir(ipath)] for ff in files1: fpath =3D "/data/infra/dir2_ren/" + file + "/" + ff=20=20=20 #keeping files opened to check it further p =3D open(fpath,"w") p1 =3D open(fpath,"w") print("I AM WRITTING") p.write("I am writting now!") new =3D "/data/infra/dir3_ren/" + file shutil.move(old,new)=20=20 print("shutil.move done") shutil.rmtree("/data/infra/dir2_ren") print("deleted dir2_ren") os.system("ls -ir /data/infra/") os.mkdir(path4) os.system("mv /data/infra/dir3_ren/* /data/infra/dir4_ren/")=20=20=20= =20=20 print("os mv command success") shutil.rmtree("/data/infra/dir3_ren") print("deleted dir3_ren") print("after dir3_ren") os.system("ls -ir /data/infra/") os.system("mkdir /data/infra/dir1") os.system("cp -rf /data/infra/dir4_ren/* /data/infra/dir1/")=20=20=20= =20=20 print("dir1 created back and copied data from dir4 to dir1") shutil.rmtree("/data/infra/dir4_ren") rtestuser@freeBSD13_2:/data/infra $ cat du.py=20 import os import time; while 1: os.system("du /") time.sleep(10) rtestuser@freeBSD13_2:/data/infra $ ls -lrt dir1/ total 0 rtestuser@freeBSD13_2:/data/infra $ ls -lrt dir4_ren/ ^C Note: In next run got stuck in shutil.move. rtestuser@freeBSD13_2:~ $ uname -r 13.2-RELEASE rtestuser@freeBSD13_2:~ $ uname -a FreeBSD freeBSD13_2.cs39 13.2-RELEASE FreeBSD 13.2-RELEASE releng/13.2-n254617-525ecfdad597 GENERIC amd64 rtestuser@freeBSD13_2:~ $ top last pid: 76719; load averages: 2.02, 1.48, 1.04=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 up 0+00:27:10= =20 18:00:02 28 processes: 3 running, 25 sleeping CPU: 19.5% user, 0.0% nice, 44.8% system, 2.4% interrupt, 33.3% idle Mem: 25M Active, 7184K Inact, 2388K Laundry, 54M Wired, 12M Buf, 3496K Free Swap: 723M Total, 13M Used, 710M Free, 1% Inuse PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 60795 root 1 103 0 22M 8908K RUN 3:23 100.00% python3.9 4544 rtestuser 1 20 0 21M 4576K select 0:00 0.00% sshd 76693 root 1 37 0 15M 2000K ufs 0:00 0.00% du 76694 root 1 36 0 15M 2000K ufs 0:00 0.00% du 76662 rtestuser 1 20 0 15M 4700K ufs 0:00 0.00% du 758 root 1 20 0 16M 1540K ttyin 0:00 0.00% csh 8447 root 1 20 0 18M 6888K wait 0:00 0.00% python3.9 764 root 1 20 0 18M 3764K wait 0:00 0.00% python3.9 702 root 1 20 0 18M 2172K select 0:00 0.00% sendmail 76699 root 1 21 0 21M 7924K select 0:00 0.00% sshd 4396 root 1 21 0 21M 4452K select 0:00 0.00% sshd 4550 rtestuser 1 21 0 13M 2464K wait 0:00 0.00% sh 596 root 1 20 0 13M 1412K select 0:00 0.00% syslogd 76703 rtestuser 1 20 0 13M 2852K wait 0:00 0.00% sh .... rtestuser@freeBSD13_2:/data/infra $ ls=20 createfiles.py dir2_ren dir3_ren du.= py=20=20 move-move-delete.py rtestuser@freeBSD13_2:/data/infra $ ls dir3_ren/ dir0 dir1 dir10 dir11 dir12 dir13 dir14 dir15 rtestuser@freeBSD13_2:/data/infra $ ls dir2_ren ^C ^C System config: 128MB RAM, CPU core -2, Disk-250GB. Thanks & regards, Shamsher --=20 You are receiving this mail because: You are the assignee for the bug.=