From owner-freebsd-fs@FreeBSD.ORG Mon Aug 6 04:00:59 2012 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 611421065670; Mon, 6 Aug 2012 04:00:59 +0000 (UTC) (envelope-from ae@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 346FA8FC14; Mon, 6 Aug 2012 04:00:59 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q7640xEi043900; Mon, 6 Aug 2012 04:00:59 GMT (envelope-from ae@freefall.freebsd.org) Received: (from ae@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q7640xDI043896; Mon, 6 Aug 2012 04:00:59 GMT (envelope-from ae) Date: Mon, 6 Aug 2012 04:00:59 GMT Message-Id: <201208060400.q7640xDI043896@freefall.freebsd.org> To: steven.hartland@FreeBSD.org, ae@FreeBSD.org, freebsd-fs@FreeBSD.org From: ae@FreeBSD.org Cc: Subject: Re: kern/161897: [zfs] [patch] zfs partition probing causing long delay at BTX loader X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Aug 2012 04:00:59 -0000 Synopsis: [zfs] [patch] zfs partition probing causing long delay at BTX loader State-Changed-From-To: open->patched State-Changed-By: ae State-Changed-When: Mon Aug 6 03:59:23 UTC 2012 State-Changed-Why: Patched in head/ with r239068. http://www.freebsd.org/cgi/query-pr.cgi?pr=161897 From owner-freebsd-fs@FreeBSD.ORG Mon Aug 6 04:35:21 2012 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2BD12106566C; Mon, 6 Aug 2012 04:35:21 +0000 (UTC) (envelope-from ae@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id F3C9F8FC0A; Mon, 6 Aug 2012 04:35:20 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q764ZKjI049666; Mon, 6 Aug 2012 04:35:20 GMT (envelope-from ae@freefall.freebsd.org) Received: (from ae@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q764ZKv3049662; Mon, 6 Aug 2012 04:35:20 GMT (envelope-from ae) Date: Mon, 6 Aug 2012 04:35:20 GMT Message-Id: <201208060435.q764ZKv3049662@freefall.freebsd.org> To: alteriks@gmail.com, ae@FreeBSD.org, freebsd-fs@FreeBSD.org From: ae@FreeBSD.org Cc: Subject: Re: kern/147560: [zfs] [boot] Booting 8.1-PRERELEASE raidz system take ages X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Aug 2012 04:35:21 -0000 Synopsis: [zfs] [boot] Booting 8.1-PRERELEASE raidz system take ages State-Changed-From-To: open->patched State-Changed-By: ae State-Changed-When: Mon Aug 6 04:35:08 UTC 2012 State-Changed-Why: Patched in head/ with r239068. http://www.freebsd.org/cgi/query-pr.cgi?pr=147560 From owner-freebsd-fs@FreeBSD.ORG Mon Aug 6 11:07:09 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 865491065688 for ; Mon, 6 Aug 2012 11:07:09 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 6F6468FC08 for ; Mon, 6 Aug 2012 11:07:09 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q76B796W021756 for ; Mon, 6 Aug 2012 11:07:09 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q76B785G021754 for freebsd-fs@FreeBSD.org; Mon, 6 Aug 2012 11:07:08 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 6 Aug 2012 11:07:08 GMT Message-Id: <201208061107.q76B785G021754@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Aug 2012 11:07:09 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/170238 fs [zfs] [panic] Panic when deleting data o kern/169945 fs [zfs] [panic] Kernel panic while importing zpool (afte o kern/169813 fs [zfs] [patch] Update with Illumos gate issues: 1796, 2 o kern/169480 fs [zfs] ZFS stalls on heavy I/O o kern/169398 fs [zfs] Can't remove file with permanent error o kern/169339 fs panic while " : > /etc/123" o kern/169319 fs [zfs] zfs resilver can't complete o kern/168947 fs [nfs] [zfs] .zfs/snapshot directory is messed up when o kern/168942 fs [nfs] [hang] nfsd hangs after being restarted (not -HU o kern/168158 fs [zfs] incorrect parsing of sharenfs options in zfs (fs o kern/167979 fs [ufs] DIOCGDINFO ioctl does not work on 8.2 file syste o kern/167977 fs [smbfs] mount_smbfs results are differ when utf-8 or U o kern/167688 fs [fusefs] Incorrect signal handling with direct_io o kern/167685 fs [zfs] ZFS on USB drive prevents shutdown / reboot o kern/167612 fs [portalfs] The portal file system gets stuck inside po o kern/167272 fs [zfs] ZFS Disks reordering causes ZFS to pick the wron o kern/167260 fs [msdosfs] msdosfs disk was mounted the second time whe o kern/167109 fs [zfs] [panic] zfs diff kernel panic Fatal trap 9: gene o kern/167105 fs [nfs] mount_nfs can not handle source exports wiht mor o kern/167067 fs [zfs] [panic] ZFS panics the server o kern/167066 fs [zfs] ZVOLs not appearing in /dev/zvol o kern/167065 fs [zfs] boot fails when a spare is the boot disk o kern/167048 fs [nfs] [patch] RELEASE-9 crash when using ZFS+NULLFS+NF o kern/166912 fs [ufs] [panic] Panic after converting Softupdates to jo o kern/166851 fs [zfs] [hang] Copying directory from the mounted UFS di o kern/166477 fs [nfs] NFS data corruption. o kern/165950 fs [ffs] SU+J and fsck problem o kern/165923 fs [nfs] Writing to NFS-backed mmapped files fails if flu o kern/165521 fs [zfs] [hang] livelock on 1 Gig of RAM with zfs when 31 o kern/165392 fs Multiple mkdir/rmdir fails with errno 31 o kern/165087 fs [unionfs] lock violation in unionfs o kern/164472 fs [ufs] fsck -B panics on particular data inconsistency o kern/164370 fs [zfs] zfs destroy for snapshot fails on i386 and sparc o kern/164261 fs [nullfs] [patch] fix panic with NFS served from NULLFS o kern/164256 fs [zfs] device entry for volume is not created after zfs o kern/164184 fs [ufs] [panic] Kernel panic with ufs_makeinode o kern/163801 fs [md] [request] allow mfsBSD legacy installed in 'swap' o kern/163770 fs [zfs] [hang] LOR between zfs&syncer + vnlru leading to o kern/163501 fs [nfs] NFS exporting a dir and a subdir in that dir to o kern/162944 fs [coda] Coda file system module looks broken in 9.0 o kern/162860 fs [zfs] Cannot share ZFS filesystem to hosts with a hyph o kern/162751 fs [zfs] [panic] kernel panics during file operations o kern/162591 fs [nullfs] cross-filesystem nullfs does not work as expe o kern/162519 fs [zfs] "zpool import" relies on buggy realpath() behavi o kern/162362 fs [snapshots] [panic] ufs with snapshot(s) panics when g o kern/161968 fs [zfs] [hang] renaming snapshot with -r including a zvo p kern/161897 fs [zfs] [patch] zfs partition probing causing long delay o kern/161864 fs [ufs] removing journaling from UFS partition fails on o bin/161807 fs [patch] add option for explicitly specifying metadata o kern/161579 fs [smbfs] FreeBSD sometimes panics when an smb share is o kern/161533 fs [zfs] [panic] zfs receive panic: system ioctl returnin o kern/161438 fs [zfs] [panic] recursed on non-recursive spa_namespace_ o kern/161424 fs [nullfs] __getcwd() calls fail when used on nullfs mou o kern/161280 fs [zfs] Stack overflow in gptzfsboot o kern/161205 fs [nfs] [pfsync] [regression] [build] Bug report freebsd o kern/161169 fs [zfs] [panic] ZFS causes kernel panic in dbuf_dirty o kern/161112 fs [ufs] [lor] filesystem LOR in FreeBSD 9.0-BETA3 o kern/160893 fs [zfs] [panic] 9.0-BETA2 kernel panic o kern/160860 fs [ufs] Random UFS root filesystem corruption with SU+J o kern/160801 fs [zfs] zfsboot on 8.2-RELEASE fails to boot from root-o o kern/160790 fs [fusefs] [panic] VPUTX: negative ref count with FUSE o kern/160777 fs [zfs] [hang] RAID-Z3 causes fatal hang upon scrub/impo o kern/160706 fs [zfs] zfs bootloader fails when a non-root vdev exists o kern/160591 fs [zfs] Fail to boot on zfs root with degraded raidz2 [r o kern/160410 fs [smbfs] [hang] smbfs hangs when transferring large fil o kern/160283 fs [zfs] [patch] 'zfs list' does abort in make_dataset_ha o kern/159930 fs [ufs] [panic] kernel core o kern/159402 fs [zfs][loader] symlinks cause I/O errors o kern/159357 fs [zfs] ZFS MAXNAMELEN macro has confusing name (off-by- o kern/159356 fs [zfs] [patch] ZFS NAME_ERR_DISKLIKE check is Solaris-s o kern/159351 fs [nfs] [patch] - divide by zero in mountnfs() o kern/159251 fs [zfs] [request]: add FLETCHER4 as DEDUP hash option o kern/159077 fs [zfs] Can't cd .. with latest zfs version o kern/159048 fs [smbfs] smb mount corrupts large files o kern/159045 fs [zfs] [hang] ZFS scrub freezes system o kern/158839 fs [zfs] ZFS Bootloader Fails if there is a Dead Disk o kern/158802 fs amd(8) ICMP storm and unkillable process. o kern/158231 fs [nullfs] panic on unmounting nullfs mounted over ufs o f kern/157929 fs [nfs] NFS slow read o kern/157399 fs [zfs] trouble with: mdconfig force delete && zfs strip o kern/157179 fs [zfs] zfs/dbuf.c: panic: solaris assert: arc_buf_remov o kern/156797 fs [zfs] [panic] Double panic with FreeBSD 9-CURRENT and o kern/156781 fs [zfs] zfs is losing the snapshot directory, p kern/156545 fs [ufs] mv could break UFS on SMP systems o kern/156193 fs [ufs] [hang] UFS snapshot hangs && deadlocks processes o kern/156039 fs [nullfs] [unionfs] nullfs + unionfs do not compose, re o kern/155615 fs [zfs] zfs v28 broken on sparc64 -current o kern/155587 fs [zfs] [panic] kernel panic with zfs p kern/155411 fs [regression] [8.2-release] [tmpfs]: mount: tmpfs : No o kern/155199 fs [ext2fs] ext3fs mounted as ext2fs gives I/O errors o bin/155104 fs [zfs][patch] use /dev prefix by default when importing o kern/154930 fs [zfs] cannot delete/unlink file from full volume -> EN o kern/154828 fs [msdosfs] Unable to create directories on external USB o kern/154491 fs [smbfs] smb_co_lock: recursive lock for object 1 p kern/154228 fs [md] md getting stuck in wdrain state o kern/153996 fs [zfs] zfs root mount error while kernel is not located o kern/153753 fs [zfs] ZFS v15 - grammatical error when attempting to u o kern/153716 fs [zfs] zpool scrub time remaining is incorrect o kern/153695 fs [patch] [zfs] Booting from zpool created on 4k-sector o kern/153680 fs [xfs] 8.1 failing to mount XFS partitions o kern/153520 fs [zfs] Boot from GPT ZFS root on HP BL460c G1 unstable o kern/153418 fs [zfs] [panic] Kernel Panic occurred writing to zfs vol o kern/153351 fs [zfs] locking directories/files in ZFS o bin/153258 fs [patch][zfs] creating ZVOLs requires `refreservation' s kern/153173 fs [zfs] booting from a gzip-compressed dataset doesn't w o kern/153126 fs [zfs] vdev failure, zpool=peegel type=vdev.too_small o kern/152022 fs [nfs] nfs service hangs with linux client [regression] o kern/151942 fs [zfs] panic during ls(1) zfs snapshot directory o kern/151905 fs [zfs] page fault under load in /sbin/zfs o bin/151713 fs [patch] Bug in growfs(8) with respect to 32-bit overfl o kern/151648 fs [zfs] disk wait bug o kern/151629 fs [fs] [patch] Skip empty directory entries during name o kern/151330 fs [zfs] will unshare all zfs filesystem after execute a o kern/151326 fs [nfs] nfs exports fail if netgroups contain duplicate o kern/151251 fs [ufs] Can not create files on filesystem with heavy us o kern/151226 fs [zfs] can't delete zfs snapshot o kern/151111 fs [zfs] vnodes leakage during zfs unmount o kern/150503 fs [zfs] ZFS disks are UNAVAIL and corrupted after reboot o kern/150501 fs [zfs] ZFS vdev failure vdev.bad_label on amd64 o kern/150390 fs [zfs] zfs deadlock when arcmsr reports drive faulted o kern/150336 fs [nfs] mountd/nfsd became confused; refused to reload n o kern/149208 fs mksnap_ffs(8) hang/deadlock o kern/149173 fs [patch] [zfs] make OpenSolaris installa o kern/149015 fs [zfs] [patch] misc fixes for ZFS code to build on Glib o kern/149014 fs [zfs] [patch] declarations in ZFS libraries/utilities o kern/149013 fs [zfs] [patch] make ZFS makefiles use the libraries fro o kern/148504 fs [zfs] ZFS' zpool does not allow replacing drives to be o kern/148490 fs [zfs]: zpool attach - resilver bidirectionally, and re o kern/148368 fs [zfs] ZFS hanging forever on 8.1-PRERELEASE o kern/148138 fs [zfs] zfs raidz pool commands freeze o kern/147903 fs [zfs] [panic] Kernel panics on faulty zfs device o kern/147881 fs [zfs] [patch] ZFS "sharenfs" doesn't allow different " p kern/147560 fs [zfs] [boot] Booting 8.1-PRERELEASE raidz system take o kern/147420 fs [ufs] [panic] ufs_dirbad, nullfs, jail panic (corrupt o kern/146941 fs [zfs] [panic] Kernel Double Fault - Happens constantly o kern/146786 fs [zfs] zpool import hangs with checksum errors o kern/146708 fs [ufs] [panic] Kernel panic in softdep_disk_write_compl o kern/146528 fs [zfs] Severe memory leak in ZFS on i386 o kern/146502 fs [nfs] FreeBSD 8 NFS Client Connection to Server s kern/145712 fs [zfs] cannot offline two drives in a raidz2 configurat o kern/145411 fs [xfs] [panic] Kernel panics shortly after mounting an f bin/145309 fs bsdlabel: Editing disk label invalidates the whole dev o kern/145272 fs [zfs] [panic] Panic during boot when accessing zfs on o kern/145246 fs [ufs] dirhash in 7.3 gratuitously frees hashes when it o kern/145238 fs [zfs] [panic] kernel panic on zpool clear tank o kern/145229 fs [zfs] Vast differences in ZFS ARC behavior between 8.0 o kern/145189 fs [nfs] nfsd performs abysmally under load o kern/144929 fs [ufs] [lor] vfs_bio.c + ufs_dirhash.c p kern/144447 fs [zfs] sharenfs fsunshare() & fsshare_main() non functi o kern/144416 fs [panic] Kernel panic on online filesystem optimization s kern/144415 fs [zfs] [panic] kernel panics on boot after zfs crash o kern/144234 fs [zfs] Cannot boot machine with recent gptzfsboot code o kern/143825 fs [nfs] [panic] Kernel panic on NFS client o bin/143572 fs [zfs] zpool(1): [patch] The verbose output from iostat o kern/143212 fs [nfs] NFSv4 client strange work ... o kern/143184 fs [zfs] [lor] zfs/bufwait LOR o kern/142878 fs [zfs] [vfs] lock order reversal o kern/142597 fs [ext2fs] ext2fs does not work on filesystems with real o kern/142489 fs [zfs] [lor] allproc/zfs LOR o kern/142466 fs Update 7.2 -> 8.0 on Raid 1 ends with screwed raid [re o kern/142306 fs [zfs] [panic] ZFS drive (from OSX Leopard) causes two o kern/142068 fs [ufs] BSD labels are got deleted spontaneously o kern/141897 fs [msdosfs] [panic] Kernel panic. msdofs: file name leng o kern/141463 fs [nfs] [panic] Frequent kernel panics after upgrade fro o kern/141305 fs [zfs] FreeBSD ZFS+sendfile severe performance issues ( o kern/141091 fs [patch] [nullfs] fix panics with DIAGNOSTIC enabled o kern/141086 fs [nfs] [panic] panic("nfs: bioread, not dir") on FreeBS o kern/141010 fs [zfs] "zfs scrub" fails when backed by files in UFS2 o kern/140888 fs [zfs] boot fail from zfs root while the pool resilveri o kern/140661 fs [zfs] [patch] /boot/loader fails to work on a GPT/ZFS- o kern/140640 fs [zfs] snapshot crash o kern/140068 fs [smbfs] [patch] smbfs does not allow semicolon in file o kern/139725 fs [zfs] zdb(1) dumps core on i386 when examining zpool c o kern/139715 fs [zfs] vfs.numvnodes leak on busy zfs p bin/139651 fs [nfs] mount(8): read-only remount of NFS volume does n o kern/139564 fs [zfs] [panic] 8.0-RC1 - Fatal trap 12 at end of shutdo o kern/139407 fs [smbfs] [panic] smb mount causes system crash if remot o kern/138662 fs [panic] ffs_blkfree: freeing free block o kern/138421 fs [ufs] [patch] remove UFS label limitations o kern/138202 fs mount_msdosfs(1) see only 2Gb o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136873 fs [ntfs] Missing directories/files on NTFS volume o kern/136865 fs [nfs] [patch] NFS exports atomic and on-the-fly atomic p kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes o kern/131441 fs [unionfs] [nullfs] unionfs and/or nullfs not combineab o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/127787 fs [lor] [ufs] Three LORs: vfslock/devfs/vfslock, ufs/vfs o bin/127270 fs fsck_msdosfs(8) may crash if BytesPerSec is zero o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file o kern/125895 fs [ffs] [panic] kernel: panic: ffs_blkfree: freeing free s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS o kern/123939 fs [msdosfs] corrupts new files o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F o kern/118912 fs [2tb] disk sizing/geometry problem with large array o kern/118713 fs [minidump] [patch] Display media size required for a k o kern/118318 fs [nfs] NFS server hangs under special circumstances o bin/118249 fs [ufs] mv(1): moving a directory changes its mtime o kern/118126 fs [nfs] [patch] Poor NFS server write performance o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o kern/117954 fs [ufs] dirhash on very large directories blocks the mac o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o conf/116931 fs lack of fsck_cd9660 prevents mounting iso images with o kern/116583 fs [ffs] [hang] System freezes for short time when using o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106107 fs [ufs] left-over fsck_snapshot after unfinished backgro o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes s bin/97498 fs [request] newfs(8) has no option to clear the first 12 o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [cd9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o bin/94810 fs fsck(8) incorrectly reports 'file system marked clean' o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88555 fs [panic] ffs_blkfree: freeing free frag on AMD 64 o kern/88266 fs [smbfs] smbfs does not implement UIO_NOCOPY and sendfi o bin/87966 fs [patch] newfs(8): introduce -A flag for newfs to enabl o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o bin/85494 fs fsck_ffs: unchecked use of cg_inosused macro etc. o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o bin/74779 fs Background-fsck checks one filesystem twice and omits o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o bin/70600 fs fsck(8) throws files away when it can't grow lost+foun o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o bin/27687 fs fsck(8) wrapper is not properly passing options to fsc o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 283 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Aug 6 15:50:04 2012 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 65433106566B for ; Mon, 6 Aug 2012 15:50:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 30DDB8FC18 for ; Mon, 6 Aug 2012 15:50:04 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q76Fo39k066402 for ; Mon, 6 Aug 2012 15:50:03 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q76Fo3BF066401; Mon, 6 Aug 2012 15:50:03 GMT (envelope-from gnats) Date: Mon, 6 Aug 2012 15:50:03 GMT Message-Id: <201208061550.q76Fo3BF066401@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: "Johannes Totz" Cc: Subject: Re: kern/167685: [zfs] ZFS on USB drive prevents shutdown / reboot X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Johannes Totz List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Aug 2012 15:50:04 -0000 The following reply was made to PR kern/167685; it has been noted by GNATS. From: "Johannes Totz" To: Cc: Subject: Re: kern/167685: [zfs] ZFS on USB drive prevents shutdown / reboot Date: Mon, 6 Aug 2012 16:34:30 +0100 Just a me too: FreeBSD fred 9.0-STABLE FreeBSD 9.0-STABLE #0 r237006: Wed Jun 13 20:38:08 BST 2012 root@fred:/usr/obj/usr/src/sys/GENERIC amd64 Setting hw.usb.no_shutdown_wait=1 shuts down properly. Without simply hangs. From owner-freebsd-fs@FreeBSD.ORG Mon Aug 6 18:45:04 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 963761065670 for ; Mon, 6 Aug 2012 18:45:04 +0000 (UTC) (envelope-from dudinskyj@gmail.com) Received: from mail-gh0-f182.google.com (mail-gh0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4B5188FC14 for ; Mon, 6 Aug 2012 18:45:04 +0000 (UTC) Received: by ghrr13 with SMTP id r13so1239435ghr.13 for ; Mon, 06 Aug 2012 11:45:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=5Em6za4YCp5OUmhlBVSE7uRhmh30a5YO3L6PUnqG+KA=; b=fOE2qfdh+wmh4ZSAIkR2J76C8xNaH08Ygn4bgfGGLgtpH7S3H+s6CbFsVA3zazjatV es8BgNZXKIUsSVeW/4fRlJc7S9mCQA9sIVxzY/FlntU6CJn48htzdhc2yp7l7PK/F/ZR 9zlYPzwv8a6V0ZHzWgHi4BGyfOyPdX2fnfyRESGUWIWBjp4Yb/TpJ8vFV7xWnFfEpSp5 36u8GVDbWRMdEVN8tZKLy4SPv30Wg7ZdUd8ahNqn5LNmoDhBolWU4LfjMXu7oVSnyFJq P10Vswgoj8Z/IDXYcZzj7GXoINwRH5yjLuOfWdCMjxgJQGDzJNNCRjFEZ0DNiWczocxj +u1w== MIME-Version: 1.0 Received: by 10.60.172.101 with SMTP id bb5mr20428048oec.44.1344278703697; Mon, 06 Aug 2012 11:45:03 -0700 (PDT) Received: by 10.182.24.168 with HTTP; Mon, 6 Aug 2012 11:45:03 -0700 (PDT) Date: Mon, 6 Aug 2012 21:45:03 +0300 Message-ID: From: Oleksandr Dudinskyi To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: How to use VOP_IOCTL? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Aug 2012 18:45:04 -0000 Hi, I am the GSoC student I need some advice in my project. http://wiki.freebsd.org/SummerOfCode2012/UDFImplementation I want to use function VOP_IOCTL() in my project, but not very informative man, introduced me to the confusion. Specifically I don't know which of the flags use. In NetBSD after call this function, request proceeds to function cdioctl () in sys/dev/scsipi/cd.c . Where the command( one of the VOP_IOCTL parameters) is selection. Could you tell me how to reach this logic in Freebsd(call cdioctl () in sys/cam /scsi/scsi_cd.c)? Or share some articles on the topic. -- Thank you, Oleksandr Dudinskyi. From owner-freebsd-fs@FreeBSD.ORG Mon Aug 6 23:55:15 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E9C5B106566C for ; Mon, 6 Aug 2012 23:55:15 +0000 (UTC) (envelope-from jusher71@yahoo.com) Received: from nm24-vm6.bullet.mail.ne1.yahoo.com (nm24-vm6.bullet.mail.ne1.yahoo.com [98.138.91.117]) by mx1.freebsd.org (Postfix) with SMTP id A6BF68FC08 for ; Mon, 6 Aug 2012 23:55:15 +0000 (UTC) Received: from [98.138.90.51] by nm24.bullet.mail.ne1.yahoo.com with NNFMP; 06 Aug 2012 23:55:14 -0000 Received: from [98.138.87.8] by tm4.bullet.mail.ne1.yahoo.com with NNFMP; 06 Aug 2012 23:55:14 -0000 Received: from [127.0.0.1] by omp1008.mail.ne1.yahoo.com with NNFMP; 06 Aug 2012 23:55:14 -0000 X-Yahoo-Newman-Property: ymail-5 X-Yahoo-Newman-Id: 731079.67046.bm@omp1008.mail.ne1.yahoo.com Received: (qmail 14786 invoked by uid 60001); 6 Aug 2012 23:55:14 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1344297314; bh=l6poIMns4EhadEOwtxM4OCByN1vxV0xcsdPU7UuZCQ0=; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:MIME-Version:Content-Type; b=iGWxJZ29zbVR/owUHsYjQ3spuhq/kLC9w26jRVgCHzpiCBhrun31SLhCZdKZ1NQufV60SWYpcexG0wlwBzs2tq/ZzATnKywliiLo6npcRaRNzOaICvt2dQC9jyFDsZVDdmzg2r1SEReakrVaidU9iKWSybvTVsVnvX++mSPXyLo= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:MIME-Version:Content-Type; b=6NVoG9nuyOVXX3G2nCtjVKcTI6lo3HqG7J3Uq04uaKnm8dHW/KxLsOAXtz58XbEPB7+0a1NYf1QaKCVqzBK5p+t40p4oC6Joi/fukF+U4FUUYht5ZspBSB789c5Laz+zyoxyM5gMtPOwUiqGZzYyt+WFgIrBBuGhNlXmOdP+5mU=; X-YMail-OSG: MszF97sVM1nZxv6Klt5fDgV9dp6XOI2phX39_4wUUTb6I3K gpCg2t98F47bUqB_7lt_jI52urCpNCG9TM8Ed77X7I0GkNliLVoOj.9Xi31K 4fyaHxdJl64BpH2deka6h2ihmormXwuFFjjLJIT2UnIDpAFNv_I.9DpHQEze iyoaiQU6cV54UiiB.MOYDBuw6dSGxtkHqb_65SpPjX5Mtwfyl9ylgbybNj79 gbkd_1HVUZ7gHTqeHxHVWtFk4hw_OJdAInwlTGN7rGJN219nYqwZfmyL8AaD xE1MD.vOFyHP35GnlY7iLt1IUqzr0PH.Jb2_474PVH8t1_bBw_YT33tuqggw C7X8RudYZsv9AcPGVxA7gl3es90sqsGU0XpU4P4XGqjfow_nv1dhwoIzWiLj Bx4xwO3aPycP0bRwj Received: from [173.164.238.34] by web121203.mail.ne1.yahoo.com via HTTP; Mon, 06 Aug 2012 16:55:14 PDT X-Mailer: YahooMailClassic/15.0.8 YahooMailWebService/0.8.120.356233 Message-ID: <1344297314.10583.YahooMailClassic@web121203.mail.ne1.yahoo.com> Date: Mon, 6 Aug 2012 16:55:14 -0700 (PDT) From: Jason Usher To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailman-Approved-At: Tue, 07 Aug 2012 02:01:12 +0000 Subject: Is securelevel, or zpool keeping me from running smartctl ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Aug 2012 23:55:16 -0000 I have a system with a running zpool, and also securelevel=2. When I run: smartctl -a /dev/da0 I get: /dev/da0: Operation not permitted Smartctl: please specify device type with the -d option. On another system, this works just fine - the disk is not being used in a zpool, AND securelevel is not enabled. So ... am I getting this error because /dev/da0 is part of a zpool, or am I getting this error because I have securelevel=2 ? Thanks. From owner-freebsd-fs@FreeBSD.ORG Tue Aug 7 05:09:25 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 50455106564A for ; Tue, 7 Aug 2012 05:09:25 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 160B08FC0A for ; Tue, 7 Aug 2012 05:09:23 +0000 (UTC) Received: by obbun3 with SMTP id un3so8870178obb.13 for ; Mon, 06 Aug 2012 22:09:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=D33tkHkgkRSOA1oazUzMcn/B/5tx3bvONTJgIa/NDKA=; b=hF76TMisXOdtPomL7tgit3hcEequGzwCctqh2hmBLo2MDGs1MSITsktdlck3d0tdlY VfZutBre75d2GgpJe7K7bhvYxo3mFgcOpep/iRLPsZfydj5RWrOO1PK7WECVnr7U/vlb DjJwtHQVAs9EG4133IqHnpCgeTzGhVbu/35ktGRcMk2AvpDLuNG1/ax8cukWuDfrZRAW 0WX6DWswut8szBRMwQmT5OpKcLnRTLr0hhoQ6pTnX4zUZDZyIscK02VwdX2/LluJyGPC TNR90HTc1B+zrAHqxioPOGG/z+kHnjW6/RlVmGXeIpheWE3bmCBHNmDloZBkD+BLkv9r ItLA== MIME-Version: 1.0 Received: by 10.182.51.37 with SMTP id h5mr22888872obo.35.1344316163359; Mon, 06 Aug 2012 22:09:23 -0700 (PDT) Received: by 10.76.167.199 with HTTP; Mon, 6 Aug 2012 22:09:23 -0700 (PDT) In-Reply-To: <1344297314.10583.YahooMailClassic@web121203.mail.ne1.yahoo.com> References: <1344297314.10583.YahooMailClassic@web121203.mail.ne1.yahoo.com> Date: Tue, 7 Aug 2012 01:09:23 -0400 Message-ID: From: Zaphod Beeblebrox To: Jason Usher Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-fs@freebsd.org Subject: Re: Is securelevel, or zpool keeping me from running smartctl ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2012 05:09:25 -0000 On Mon, Aug 6, 2012 at 7:55 PM, Jason Usher wrote: > When I run: > > smartctl -a /dev/da0 > > I get: > > /dev/da0: Operation not permitted > Smartctl: please specify device type with the -d option. In my experience, this is often due to a "path" that doesn't support smartctl messages... like USB or a RAID controller that doesn't implement them. From owner-freebsd-fs@FreeBSD.ORG Tue Aug 7 07:38:03 2012 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 83CAC106566C; Tue, 7 Aug 2012 07:38:03 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 566A08FC08; Tue, 7 Aug 2012 07:38:03 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q777c3As009840; Tue, 7 Aug 2012 07:38:03 GMT (envelope-from mm@freefall.freebsd.org) Received: (from mm@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q777c39Z009834; Tue, 7 Aug 2012 07:38:03 GMT (envelope-from mm) Date: Tue, 7 Aug 2012 07:38:03 GMT Message-Id: <201208070738.q777c39Z009834@freefall.freebsd.org> To: araujo@FreeBSD.org, mm@FreeBSD.org, freebsd-fs@FreeBSD.org From: mm@FreeBSD.org Cc: Subject: Re: kern/169813: [zfs] [patch] Update with Illumos gate issues: 1796, 2871, 2903, 2957 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2012 07:38:03 -0000 Synopsis: [zfs] [patch] Update with Illumos gate issues: 1796, 2871, 2903, 2957 State-Changed-From-To: open->closed State-Changed-By: mm State-Changed-When: Tue Aug 7 07:38:02 UTC 2012 State-Changed-Why: Committed in r238422 http://www.freebsd.org/cgi/query-pr.cgi?pr=169813 From owner-freebsd-fs@FreeBSD.ORG Tue Aug 7 13:34:15 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8CC42106564A for ; Tue, 7 Aug 2012 13:34:15 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id DEF758FC08 for ; Tue, 7 Aug 2012 13:34:14 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q77DXmQk021340; Tue, 7 Aug 2012 16:33:49 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q77DXYVX036416; Tue, 7 Aug 2012 16:33:34 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q77DXY6e036415; Tue, 7 Aug 2012 16:33:34 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 7 Aug 2012 16:33:34 +0300 From: Konstantin Belousov To: Oleksandr Dudinskyi Message-ID: <20120807133334.GL2676@deviant.kiev.zoral.com.ua> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="suFzMqjmaXLIoFqW" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-fs@freebsd.org Subject: Re: How to use VOP_IOCTL? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2012 13:34:15 -0000 --suFzMqjmaXLIoFqW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Aug 06, 2012 at 09:45:03PM +0300, Oleksandr Dudinskyi wrote: > Hi, > I am the GSoC student I need some advice in my project. > http://wiki.freebsd.org/SummerOfCode2012/UDFImplementation > I want to use function VOP_IOCTL() in my project, but not very informative > man, introduced me to the confusion. Specifically I don't know which of t= he > flags use. In NetBSD after call this function, request proceeds to functi= on > cdioctl () in sys/dev/scsipi/cd.c . Where the command( one of the > VOP_IOCTL parameters) is selection. Could you tell me how to reach this > logic in Freebsd(call cdioctl () in sys/cam /scsi/scsi_cd.c)? Or share > some articles on the topic. VOP_IOCTL() is just a forwarder of the generic ioctl arguments to an implementation of the VOP in the vnode' filesystem. You claim that you 'want to use VOP_IOCTL', but you later describes that you want to=20 execute some CDROM-specific ioctl's. These things are not related. My first question would be why do you need to execute device-specific ioctl's in the filesystem code ? Do you plan for your UDF implementation to be usable with e.g. md-mounted images ? Second, if my guess is right and you want to execute device-specific ioctl on the special device specified as your mount source, you probably need the geom providers' geom ioctl method. You should have performed some initial manipulations with the provider anyway at the mount time to get access to i/o operations on it. Then provider->geom->ioctl is the pointer to the provider ioctl implementation. Might be, look at the g_dev_ioctl(9) for start. --suFzMqjmaXLIoFqW Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlAhGS4ACgkQC3+MBN1Mb4i4bgCgsN3S6loAbdVhgUm29/V616V5 ZtgAoKwtKOgml9X60sDJGrBYwk/oFMIL =tEd8 -----END PGP SIGNATURE----- --suFzMqjmaXLIoFqW-- From owner-freebsd-fs@FreeBSD.ORG Wed Aug 8 04:41:50 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1EAA2106566B for ; Wed, 8 Aug 2012 04:41:50 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from gw.catspoiler.org (gw.catspoiler.org [75.1.14.242]) by mx1.freebsd.org (Postfix) with ESMTP id F3C518FC0A for ; Wed, 8 Aug 2012 04:41:49 +0000 (UTC) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.13.3/8.13.3) with ESMTP id q784OEfY051025 for ; Tue, 7 Aug 2012 21:24:18 -0700 (PDT) (envelope-from truckman@FreeBSD.org) Message-Id: <201208080424.q784OEfY051025@gw.catspoiler.org> Date: Tue, 7 Aug 2012 21:24:14 -0700 (PDT) From: Don Lewis To: freebsd-fs@FreeBSD.org MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Cc: Subject: ZFS questions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Aug 2012 04:41:50 -0000 I've got a couple of questions about a raidz array that I'm putting together. Capacity is more important to me than speed, but I don't want to do anything too stupid. The fine manual says that using whole disks is preferable to using slices because drive write-caching is enabled if the entire drive is dedicated to ZFS, which would break things if the drive also contained a UFS slice. Does this really matter much if NCQ is available? Each drive will be dedicated to ZFS, but I'm planning on using GPT to slightly undersize the the ZFS slice on each drive to avoid any potential issues of installing replacement drives that are slightly smaller than the original drives. I'm slowly accumulating the drives over time for both budgetary reasons and to also try to reduce the chances of multiple near-simultaneous failures of drives from the same manufacturing batch. I'd like to get the array up and running before I have all the drives, but unfortunately ZFS doesn't allow new drives to be added to an existing raidz vdev to increase its capacity. I do have some smaller drives and I was thinking about pairing those up with gconcat or gstripe and configuring the ZFS pool with the concatenated/striped pairs. I know this isn't recommended, but it seems to me like zpool create would accept this. What concerns me is happens on reboot when ZFS goes searching for all of the components of its pool. Will it stumble across its metadata on the first of the two concatenated pairs and try to add that individual drive to the pool instead of the pair? From owner-freebsd-fs@FreeBSD.ORG Wed Aug 8 08:21:19 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 21DA81065670 for ; Wed, 8 Aug 2012 08:21:19 +0000 (UTC) (envelope-from ronald-freebsd8@klop.yi.org) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.78]) by mx1.freebsd.org (Postfix) with ESMTP id A10BB8FC12 for ; Wed, 8 Aug 2012 08:21:18 +0000 (UTC) Received: from smtp.greenhost.nl ([213.108.104.138]) by smarthost1.greenhost.nl with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.69) (envelope-from ) id 1Sz1GP-0004ch-VH for freebsd-fs@freebsd.org; Wed, 08 Aug 2012 10:05:10 +0200 Received: from [81.21.138.17] (helo=ronaldradial.versatec.local) by smtp.greenhost.nl with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from ) id 1Sz1GQ-0000FC-BY for freebsd-fs@freebsd.org; Wed, 08 Aug 2012 10:05:10 +0200 Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes To: freebsd-fs@freebsd.org References: <201208080424.q784OEfY051025@gw.catspoiler.org> Date: Wed, 08 Aug 2012 10:05:10 +0200 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: "Ronald Klop" Message-ID: In-Reply-To: <201208080424.q784OEfY051025@gw.catspoiler.org> User-Agent: Opera Mail/12.01 (Win32) X-Virus-Scanned: by clamav at smarthost1.samage.net X-Spam-Level: / X-Spam-Score: 0.0 X-Spam-Status: No, score=0.0 required=5.0 tests=BAYES_50 autolearn=disabled version=3.2.5 X-Scan-Signature: 5a1627636b35b65657045ef62631cd80 Subject: Re: ZFS questions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Aug 2012 08:21:19 -0000 On Wed, 08 Aug 2012 06:24:14 +0200, Don Lewis wrote: > I've got a couple of questions about a raidz array that I'm putting > together. Capacity is more important to me than speed, but I don't want > to do anything too stupid. > > The fine manual says that using whole disks is preferable to using > slices because drive write-caching is enabled if the entire drive is > dedicated to ZFS, which would break things if the drive also contained a > UFS slice. Does this really matter much if NCQ is available? Each > drive will be dedicated to ZFS, but I'm planning on using GPT to > slightly undersize the the ZFS slice on each drive to avoid any > potential issues of installing replacement drives that are slightly > smaller than the original drives. Solaris does/did this. FreeBSD does not disable the write-cache if you don't use the whole disk. > I'm slowly accumulating the drives over time for both budgetary reasons > and to also try to reduce the chances of multiple near-simultaneous > failures of drives from the same manufacturing batch. I'd like to get > the array up and running before I have all the drives, but unfortunately > ZFS doesn't allow new drives to be added to an existing raidz vdev to > increase its capacity. I do have some smaller drives and I was thinking > about pairing those up with gconcat or gstripe and configuring the ZFS > pool with the concatenated/striped pairs. I know this isn't > recommended, but it seems to me like zpool create would accept this. > What concerns me is happens on reboot when ZFS goes searching for all of > the components of its pool. Will it stumble across its metadata on the > first of the two concatenated pairs and try to add that individual drive > to the pool instead of the pair? I don't know. Somebody else might answer this. Ronald. From owner-freebsd-fs@FreeBSD.ORG Wed Aug 8 13:08:40 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E17411065676 for ; Wed, 8 Aug 2012 13:08:40 +0000 (UTC) (envelope-from break19@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 992B48FC08 for ; Wed, 8 Aug 2012 13:08:40 +0000 (UTC) Received: by yenl7 with SMTP id l7so881094yen.13 for ; Wed, 08 Aug 2012 06:08:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:subject:message-id:in-reply-to:references:x-mailer :mime-version:content-type:content-transfer-encoding; bh=S//+VjPeX68y5eD6lRGCie3MR4t3nn052ljhuyEIQ+k=; b=GumHmZHGdMHYb1XP7S24GN+WaZsiTl9bozGQBzifsTpr4ZLhoNsHwg1UvW6Qu986A1 ER1BARLkyMjE4wHaC69PAv8YqMrSKkMVcOAMWq9zS7bOwZwy+BzF6k8svR5VMeEasEVe H+3CsAbLqjBXrYfMs7eL4Nk64XTv7Nm5qp6HrbyIV0I4fv2Eh5o/+VlpAX1MFJasghN7 A9SQVNIQp19utolsLzu0Cl4HkMlmzXNZasVR2PQs4LpJTyrdUsHev69gBxIWhK+JhB8A s1uzxcbFp59qKo5feYQQy1SbehGTYkLj08jBL4Gt07qYgApX3HYvFkimefn8euuiJ6fk 23dw== Received: by 10.50.87.198 with SMTP id ba6mr892451igb.22.1344431313831; Wed, 08 Aug 2012 06:08:33 -0700 (PDT) Received: from AL-MB-MWS0001.mediacomcorp.com ([173.156.229.158]) by mx.google.com with ESMTPS id dk7sm4796139igb.10.2012.08.08.06.08.32 (version=SSLv3 cipher=OTHER); Wed, 08 Aug 2012 06:08:33 -0700 (PDT) Date: Wed, 8 Aug 2012 08:08:29 -0500 From: Chuck Burns To: freebsd-fs@freebsd.org Message-Id: <20120808080829.534e6e16.break19@gmail.com> In-Reply-To: References: <201208080424.q784OEfY051025@gw.catspoiler.org> X-Mailer: Sylpheed 3.1.3 (GTK+ 2.10.14; i686-pc-mingw32) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: ZFS questions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Aug 2012 13:08:41 -0000 On Wed, 08 Aug 2012 10:05:10 +0200 "Ronald Klop" wrote: > > I'm slowly accumulating the drives over time for both budgetary reasons > > and to also try to reduce the chances of multiple near-simultaneous > > failures of drives from the same manufacturing batch. I'd like to get > > the array up and running before I have all the drives, but unfortunately > > ZFS doesn't allow new drives to be added to an existing raidz vdev to > > increase its capacity. I do have some smaller drives and I was thinking > > about pairing those up with gconcat or gstripe and configuring the ZFS > > pool with the concatenated/striped pairs. I know this isn't > > recommended, but it seems to me like zpool create would accept this. > > What concerns me is happens on reboot when ZFS goes searching for all of > > the components of its pool. Will it stumble across its metadata on the > > first of the two concatenated pairs and try to add that individual drive > > to the pool instead of the pair? > > I don't know. Somebody else might answer this. > > Ronald. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" Wouldn't it work to just use 1 new, large drive, along with the rest of the smaller drives, simply replacing them with the large drives as you get them? the -size- of the vdevs can change.. and from my limited testing, it seems you can upgrade space, not by adding more drives, but by replacing existing drives in the vdev with larger ones, then resilvering.. -- Chuck Burns From owner-freebsd-fs@FreeBSD.ORG Wed Aug 8 15:35:18 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 33E33106566B for ; Wed, 8 Aug 2012 15:35:18 +0000 (UTC) (envelope-from bgold@simons-rock.edu) Received: from hedwig.simons-rock.edu (hedwig.simons-rock.edu [208.81.88.14]) by mx1.freebsd.org (Postfix) with ESMTP id 0C2088FC0C for ; Wed, 8 Aug 2012 15:35:17 +0000 (UTC) Received: from behemoth (behemoth.simons-rock.edu [10.30.2.44]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by hedwig.simons-rock.edu (Postfix) with ESMTPSA id D160514B for ; Wed, 8 Aug 2012 11:28:01 -0400 (EDT) From: "Brian Gold" To: Date: Wed, 8 Aug 2012 11:27:51 -0400 Message-ID: <0c8801cd757a$601018e0$20304aa0$@simons-rock.edu> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 14.0 Thread-Index: Ac11eXkMakf2B490Qymv7m3VVLtyfw== Content-Language: en-us Subject: undoing zfs deduplication X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Aug 2012 15:35:18 -0000 I've got a system running 9.0-release w/ a zfs v28 pool. Within that pool I have 3 datasets, two of which have deduplication enabled. I've recently been having a lot of performance issues with deduplication and have determined that I need far more ram that I currently have in order to support dedupe. I don't have the budget for the ram necessary so I would like to move away from deduplication. I'm aware that you can't simply turn dedupe off, you need to completely nuke the filesystem. What I'm wondering is, would it be possible for me to create new datasets within the same pool (I have a ton of available space) and use a combination of "zfs send" & "zfs receive" to migrate my deduped datasets and all of their snapshots (daily, weekly, & monthly) over to the new dataset? Brian Gold System Administrator Bard College at Simon's Rock From owner-freebsd-fs@FreeBSD.ORG Wed Aug 8 15:41:45 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 700CF106566B for ; Wed, 8 Aug 2012 15:41:45 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id E8EAD8FC17 for ; Wed, 8 Aug 2012 15:41:44 +0000 (UTC) Received: by lbbgk8 with SMTP id gk8so676770lbb.13 for ; Wed, 08 Aug 2012 08:41:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=14Kf5avZ5vhWwHcIjcuZ3u6pcP4dx+Mrj6WEH2tghB0=; b=0O9zRdhqGmh5HsBgCdGiVEhKwcCq3/KP3GzzLOjSNCQtZU5bJhgh5XhdgRWet+FgWY vLC5fWFGSWnBC8xt5WPfwfYlxYo4Wsv+rQvztaR8k4fkVsy9QNk1Px4m9d+JdHGec0dc 0AQtbXsGmrUlaYt1hDmFLmnyhU+zuriYFLqAeCbDq/NNLaVZOKBc4Uu00YuolBLOKnBD 6Ma/C4tpXFFkYmeKqoby8XypiVXWgBjEic/6ac6Sqh0xJ591wdeU/0xJes3WaQU/PouC jhbZJw+dVNLVrTLqxsXNbRkTQ6XD4ksy13kDZqIXtVDNmFdmE7Zyxl1UixUxSYS1DcBd d+wQ== MIME-Version: 1.0 Received: by 10.152.105.173 with SMTP id gn13mr18587587lab.20.1344440503541; Wed, 08 Aug 2012 08:41:43 -0700 (PDT) Received: by 10.114.16.229 with HTTP; Wed, 8 Aug 2012 08:41:43 -0700 (PDT) In-Reply-To: <0c8801cd757a$601018e0$20304aa0$@simons-rock.edu> References: <0c8801cd757a$601018e0$20304aa0$@simons-rock.edu> Date: Wed, 8 Aug 2012 08:41:43 -0700 Message-ID: From: Freddie Cash To: Brian Gold Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs@freebsd.org Subject: Re: undoing zfs deduplication X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Aug 2012 15:41:45 -0000 On Wed, Aug 8, 2012 at 8:27 AM, Brian Gold wrote: > I've got a system running 9.0-release w/ a zfs v28 pool. Within that pool I have 3 datasets, two of which have deduplication > enabled. I've recently been having a lot of performance issues with deduplication and have determined that I need far more ram that > I currently have in order to support dedupe. I don't have the budget for the ram necessary so I would like to move away from > deduplication. I'm aware that you can't simply turn dedupe off, you need to completely nuke the filesystem. > > What I'm wondering is, would it be possible for me to create new datasets within the same pool (I have a ton of available space) and > use a combination of "zfs send" & "zfs receive" to migrate my deduped datasets and all of their snapshots (daily, weekly, & monthly) > over to the new dataset? Yes, that is the only option for "un-deduping" a filesystem. zfs send/recv from the deduped filesystem to one with dedup=off. Then delete the deduped filesystem. Note: a "zfs destroy" will use a lot of RAM as it has to go through an update all the DDT entries. You may have to manually delete individual snapshots, and then manually delete individual directories in the filesystem, before destroying the actual filesystem. You may run into a situation where you don't have enough RAM/ARC to destroy a deduped filesystem. -- Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Wed Aug 8 15:55:15 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D1DBB1065670 for ; Wed, 8 Aug 2012 15:55:15 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 5DF938FC17 for ; Wed, 8 Aug 2012 15:55:15 +0000 (UTC) Received: by eeke52 with SMTP id e52so310271eek.13 for ; Wed, 08 Aug 2012 08:55:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=QP4qft1ypxwnqECzseUp3i49V/7CldhO1qmBt5CtReo=; b=UdG1Vmiso17TKt1EYsE2GwJkuUJwZxwwxsBrn9H7Yy57zuy3fJlefa1g0SFCx18E3H mV6nSBO0F8omdAYVyevxCZ3F6NDCu+Y4zEIvtWRii03DuDryaWXA2ackABKkfgWEnvhD bc6o+HR9mRST/ekcyy9zgbDt1zW8+YG8Ly9rjOzdfRjX4TZYf5jvk08a88EcE8z5yrGW fIoc7VkW3W8nphAppWLhjTWwQ4lT+acVxZoNSwbSkn5lcXKT9N9dgk2Wn0d/fAGVvcYA 5IQaMdsg3AorhNF+VPelUsdF9L9kVJQv7yO6uumUgguDz+3aYkPhk25BUQnr9xUANYhw EkQw== Received: by 10.14.173.71 with SMTP id u47mr450049eel.22.1344441314337; Wed, 08 Aug 2012 08:55:14 -0700 (PDT) Received: from imba-brutale.totalterror.net ([93.152.184.10]) by mx.google.com with ESMTPS id e7sm29632708eep.2.2012.08.08.08.55.12 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 08 Aug 2012 08:55:13 -0700 (PDT) Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1485\)) Content-Type: text/plain; charset=us-ascii From: Nikolay Denev In-Reply-To: <0c8801cd757a$601018e0$20304aa0$@simons-rock.edu> Date: Wed, 8 Aug 2012 18:55:14 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: <63850C98-E472-40B9-8E7F-93FD6AC6B95E@gmail.com> References: <0c8801cd757a$601018e0$20304aa0$@simons-rock.edu> To: Brian Gold X-Mailer: Apple Mail (2.1485) Cc: freebsd-fs@freebsd.org Subject: Re: undoing zfs deduplication X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Aug 2012 15:55:15 -0000 On Aug 8, 2012, at 6:27 PM, Brian Gold wrote: > I've got a system running 9.0-release w/ a zfs v28 pool. Within that = pool I have 3 datasets, two of which have deduplication > enabled. I've recently been having a lot of performance issues with = deduplication and have determined that I need far more ram that > I currently have in order to support dedupe. I don't have the budget = for the ram necessary so I would like to move away from > deduplication. I'm aware that you can't simply turn dedupe off, you = need to completely nuke the filesystem.=20 >=20 > What I'm wondering is, would it be possible for me to create new = datasets within the same pool (I have a ton of available space) and > use a combination of "zfs send" & "zfs receive" to migrate my deduped = datasets and all of their snapshots (daily, weekly, & monthly) > over to the new dataset?=20 >=20 > Brian Gold > System Administrator > Bard College at Simon's Rock >=20 >=20 >=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" I believe simply disabling dedup should be enough. Some blocks still will be just a "reference" to the original block = because of the previously enabled deduplication, but this won't use = additional memory. New files will be written without being deduped. If you want to de-dedup the files you can probably just "rsync $source = $source.tmp && mv $source.tmp $source" in a loop to rewrite them.= From owner-freebsd-fs@FreeBSD.ORG Wed Aug 8 16:09:04 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 97154106564A for ; Wed, 8 Aug 2012 16:09:04 +0000 (UTC) (envelope-from bgold@simons-rock.edu) Received: from hedwig.simons-rock.edu (hedwig.simons-rock.edu [208.81.88.14]) by mx1.freebsd.org (Postfix) with ESMTP id 6A1FD8FC0A for ; Wed, 8 Aug 2012 16:09:04 +0000 (UTC) Received: from behemoth (behemoth.simons-rock.edu [10.30.2.44]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by hedwig.simons-rock.edu (Postfix) with ESMTPSA id E2DEB13E; Wed, 8 Aug 2012 12:09:03 -0400 (EDT) From: "Brian Gold" To: "'Freddie Cash'" References: <0c8801cd757a$601018e0$20304aa0$@simons-rock.edu> In-Reply-To: Date: Wed, 8 Aug 2012 12:08:53 -0400 Message-ID: <0c9d01cd7580$1b8ecee0$52ac6ca0$@simons-rock.edu> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQJsV7V78kgQS/TPjmzlbMJpfEPggQF1n0LrlgadxrA= Content-Language: en-us Cc: freebsd-fs@freebsd.org Subject: RE: undoing zfs deduplication X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Aug 2012 16:09:04 -0000 > Yes, that is the only option for "un-deduping" a filesystem. >=20 > zfs send/recv from the deduped filesystem to one with dedup=3Doff. = Then delete the deduped filesystem. >=20 > Note: a "zfs destroy" will use a lot of RAM as it has to go through = an update all the DDT entries. You may have to manually delete > individual snapshots, and then manually delete individual directories = in the filesystem, before destroying the actual filesystem. You > may run into a situation where you don't have enough RAM/ARC to = destroy a deduped filesystem. >=20 > -- > Freddie Cash > fjwcash@gmail.com >From what I've read so far, it looks like a "zfs send -R" would send the = filesystem and all of the snapshots I've made. So would something like = this work to move the duped filesystem and all of its snapshots over to = a new undeduped filesystem: "zfs send -R backup/duped | zfs receive -duv = backup/deduped" ? From owner-freebsd-fs@FreeBSD.ORG Wed Aug 8 16:24:27 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id ADC9F106564A for ; Wed, 8 Aug 2012 16:24:27 +0000 (UTC) (envelope-from ef@math.uni-bonn.de) Received: from ems.math.uni-bonn.de (ems.math.uni-bonn.de [131.220.132.179]) by mx1.freebsd.org (Postfix) with ESMTP id 7318F8FC14 for ; Wed, 8 Aug 2012 16:24:26 +0000 (UTC) Received: from trav.math.uni-bonn.de (pD9E80754.dip0.t-ipconnect.de [217.232.7.84]) by ems.math.uni-bonn.de (Postfix) with ESMTPSA id DE70FBC9A5 for ; Wed, 8 Aug 2012 18:24:19 +0200 (CEST) Date: Wed, 8 Aug 2012 18:24:17 +0200 From: Edgar =?iso-8859-1?B?RnXf?= To: freebsd-fs@freebsd.org Message-ID: <20120808162417.GM8825@trav.math.uni-bonn.de> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: Book on FFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Aug 2012 16:24:27 -0000 In May, I asked: > Is there a book (or other documentation) about FFS giving more details > than the 4.4BSD bible? Having received no reply up to now I deduce that probably no-one round here needs such a thing because, when in doubt, you can always ask Kirk Himself. From owner-freebsd-fs@FreeBSD.ORG Wed Aug 8 16:32:28 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 26075106564A for ; Wed, 8 Aug 2012 16:32:28 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9B1CD8FC0A for ; Wed, 8 Aug 2012 16:32:27 +0000 (UTC) Received: by lbbgk8 with SMTP id gk8so714224lbb.13 for ; Wed, 08 Aug 2012 09:32:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=0dPW2h5n9W4mnoEDpjNX0Ga+A70iy1gKa6fMdq8tnYw=; b=xBlgJrlvd4ngg5EjHgEx+MPrRqlqLqeAGy3o3imbMCOmQlJfHJl+J7uJG0Dd8cShPF PErjaz+5BQir/oVttSEVWO4KaTEqaXcaaNe0oDSoIykVhrrMgu5E2ref6oQUJgk5YBiv pvuWzwLdTP/kjGYnZC3G8OOf27cOJkZ5Z4J8z6s74k8a+hsn6RV75kISk6C31O4clI4d D81bpAXyAepGcEA9PMIwKOotC9N2SoAqOURKSQHp626llYLI5aEP/ZdaKQ6QN8YLQde0 ZqohRHETDMmsithYlK9siOoz/f+DaXOo+QYzeBYKh6HueMyv0cbZas9gV1aT6GSFSeDW dMDw== MIME-Version: 1.0 Received: by 10.152.111.71 with SMTP id ig7mr18632350lab.28.1344443546465; Wed, 08 Aug 2012 09:32:26 -0700 (PDT) Received: by 10.114.16.229 with HTTP; Wed, 8 Aug 2012 09:32:26 -0700 (PDT) In-Reply-To: <0c9d01cd7580$1b8ecee0$52ac6ca0$@simons-rock.edu> References: <0c8801cd757a$601018e0$20304aa0$@simons-rock.edu> <0c9d01cd7580$1b8ecee0$52ac6ca0$@simons-rock.edu> Date: Wed, 8 Aug 2012 09:32:26 -0700 Message-ID: From: Freddie Cash To: Brian Gold Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: undoing zfs deduplication X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Aug 2012 16:32:28 -0000 On Wed, Aug 8, 2012 at 9:08 AM, Brian Gold wrote: >> Yes, that is the only option for "un-deduping" a filesystem. >> >> zfs send/recv from the deduped filesystem to one with dedup=3Doff. Then= delete the deduped filesystem. >> >> Note: a "zfs destroy" will use a lot of RAM as it has to go through an = update all the DDT entries. You may have to manually delete >> individual snapshots, and then manually delete individual directories in= the filesystem, before destroying the actual filesystem. You >> may run into a situation where you don't have enough RAM/ARC to destroy = a deduped filesystem. >> >> -- >> Freddie Cash >> fjwcash@gmail.com > > From what I've read so far, it looks like a "zfs send -R" would send the = filesystem and all of the snapshots I've made. So would something like this= work to move the duped filesystem and all of its snapshots over to a new u= ndeduped filesystem: "zfs send -R backup/duped | zfs receive -duv backup/de= duped" ? Yes. That will create a new filesystem and snapshots of the non-deduped da= ta. You would still have to delete the deduped filesystem, though, to clear out the DDT and remove the extra RAM requirements of dedupe. --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Thu Aug 9 02:47:48 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EE5A2106566B for ; Thu, 9 Aug 2012 02:47:48 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from gw.catspoiler.org (gw.catspoiler.org [75.1.14.242]) by mx1.freebsd.org (Postfix) with ESMTP id B0F908FC12 for ; Thu, 9 Aug 2012 02:47:48 +0000 (UTC) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.13.3/8.13.3) with ESMTP id q792ldPo053970; Wed, 8 Aug 2012 19:47:43 -0700 (PDT) (envelope-from truckman@FreeBSD.org) Message-Id: <201208090247.q792ldPo053970@gw.catspoiler.org> Date: Wed, 8 Aug 2012 19:47:39 -0700 (PDT) From: Don Lewis To: break19@gmail.com In-Reply-To: <20120808080829.534e6e16.break19@gmail.com> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Cc: freebsd-fs@FreeBSD.org Subject: Re: ZFS questions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Aug 2012 02:47:49 -0000 On 8 Aug, Chuck Burns wrote: > On Wed, 08 Aug 2012 10:05:10 +0200 > "Ronald Klop" wrote: > >> > I'm slowly accumulating the drives over time for both budgetary >> > reasons and to also try to reduce the chances of multiple >> > near-simultaneous failures of drives from the same manufacturing >> > batch. I'd like to get the array up and running before I have all >> > the drives, but unfortunately ZFS doesn't allow new drives to be >> > added to an existing raidz vdev to increase its capacity. I do >> > have some smaller drives and I was thinking about pairing those up >> > with gconcat or gstripe and configuring the ZFS pool with the >> > concatenated/striped pairs. I know this isn't recommended, but it >> > seems to me like zpool create would accept this. What concerns me >> > is happens on reboot when ZFS goes searching for all of the >> > components of its pool. Will it stumble across its metadata on the >> > first of the two concatenated pairs and try to add that individual >> > drive to the pool instead of the pair? >> >> I don't know. Somebody else might answer this. > Wouldn't it work to just use 1 new, large drive, along with the rest > of the smaller drives, simply replacing them with the large drives as > you get them? the -size- of the vdevs can change.. and from my > limited testing, it seems you can upgrade space, not by adding more > drives, but by replacing existing drives in the vdev with larger ones, > then resilvering.. Yes, I'll actually be going through this stage first. The problem is that the extra space won't be available until all of the drives have been upgraded. The capacity is limited to (N-P) times the size of the smallest drive. My objective is to build an 11 drive raidz3 array using 4 TB drives. I'll start off with a mixture of 2 TB and 4 TB drives and the 2 TB drives will limit my initial capacity to ~16 TB. I've got some more 4 TB drives in a mixed JBOD array on a linux machine. As I move the data off each of those drives, I'll move them over to the ZFS array, replacing some of the 2 TB drives. Whenever I free up a 2 TB drive, I can concatenate it with one of the 2 TB drives already in the array. Once all of the 2 TB drives are paired, then my array capacity will double. Then I can start replacing pairs of 2 TB drives with 4 TB drives. This is going to take a while ... When I first put bring up the array, I'll have just enough disks to configure it and bring it online. After it is up, I'll start migrating data to it from a Linux JBOD array which contains a wide variety of drives. It's got some of the same large drives that I'm using for the ZFS array. I'll migrate the data off of those drives first and as they get freed up, I'll migrate them to the ZFS array, replacing the smaller drives in the ZFS array one at a time and resilvering. Unfortunately, the available space won't increase when I do this. From owner-freebsd-fs@FreeBSD.ORG Thu Aug 9 09:06:26 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 27452106566C for ; Thu, 9 Aug 2012 09:06:26 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id BD0C48FC0A for ; Thu, 9 Aug 2012 09:06:25 +0000 (UTC) Received: from lstewart.caia.swin.edu.au (lstewart.caia.swin.edu.au [136.186.229.95]) by lauren.room52.net (Postfix) with ESMTPSA id 30C0E7E820; Thu, 9 Aug 2012 18:57:24 +1000 (EST) Message-ID: <50237B73.9040301@freebsd.org> Date: Thu, 09 Aug 2012 18:57:23 +1000 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120702 Thunderbird/13.0.1 MIME-Version: 1.0 To: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lauren.room52.net Subject: gpart rewrites pmbr in a way which breaks Win 7 EFI bootloader X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Aug 2012 09:06:26 -0000 Hi all, I recently got a new HP Compaq Elite 8200 work desktop which has a hybrid UEFI and regular BIOS arrangement. I want to dual-boot Win 7 64bit Enterprise via EFI and FreeBSD 9-STABLE via BIOS+gptzfsboot (until such time as we gain EFI support of course) on the machine and ran into a problem. I finally got around to analysing the issue in detail and wrote it up here: http://people.freebsd.org/~lstewart/misc/gpart_debug/ The summary is that FreeBSD's gpart rewrites the protective MBR with 0x80 (active) in the single MBR partition table entry, which makes the MS EFI bootloader unhappy and it will stop booting Win 7. The gpart command I ran to trigger the behaviour was "gpart modify -i 5 -t freebsd-zfs ", where partition 5 was already created in Win 7 and manually set using Win 7's diskpart utility to type freebsd-zfs i.e. the gpart command should have been a NOP. Indeed, the GPT remains unchanged, but the pmbr gets changed, which is what breaks Win 7 booting. After identifying the cause of the problem and a workaround (please see the README.txt at the above URL for full details and pre/post gpart dumps of the MBR+GPT), I have the following questions: - Should gpart be writing 0x80 (active) in the protective MBR entry? - Why is Windows EFI bootloader so sensitive to 0x80 in pmbr? - Should gpart be silently rewriting the protective MBR entry at all when only asked to make changes to the GPT? Thanks in advance for any insights and help. Cheers, Lawrence From owner-freebsd-fs@FreeBSD.ORG Thu Aug 9 10:57:43 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CEAD4106566B; Thu, 9 Aug 2012 10:57:43 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward4.mail.yandex.net (forward4.mail.yandex.net [IPv6:2a02:6b8:0:602::4]) by mx1.freebsd.org (Postfix) with ESMTP id 3A7D38FC1B; Thu, 9 Aug 2012 10:57:43 +0000 (UTC) Received: from smtp3.mail.yandex.net (smtp3.mail.yandex.net [77.88.46.103]) by forward4.mail.yandex.net (Yandex) with ESMTP id BE0021BC1BDE; Thu, 9 Aug 2012 14:57:41 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1344509861; bh=R8TuQ+0BEi5A+B5fwi7f4CfO8y7u9soSrse9EKBvdx8=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type; b=axRggd5l9vpjBy4X4VLzETqw6ecWQYNFYKmeEtwWfFZ7PV3/h8LDXjXsne2DXWi4g 8I5hbh+BWNMa61R1lzn+LGLRAYReYeVxPGXgj33Y0pG61D/TcI/NaR+2kfpOPaUHFr x6cNQNTqgHk6hYjCh1Db0bKjDx0afepVif+UZG7w= Received: from smtp3.mail.yandex.net (localhost [127.0.0.1]) by smtp3.mail.yandex.net (Yandex) with ESMTP id 88CB71BA0344; Thu, 9 Aug 2012 14:57:41 +0400 (MSK) Received: from mail.kirov.so-ups.ru (mail.kirov.so-ups.ru [178.74.170.1]) by smtp3.mail.yandex.net (nwsmtp/Yandex) with ESMTP id vaJ8s03S-vdJ8GRnS; Thu, 9 Aug 2012 14:57:41 +0400 X-Yandex-Rcpt-Suid: lstewart@freebsd.org X-Yandex-Rcpt-Suid: freebsd-fs@freebsd.org X-Yandex-Rcpt-Suid: marcel@FreeBSD.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1344509861; bh=R8TuQ+0BEi5A+B5fwi7f4CfO8y7u9soSrse9EKBvdx8=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:X-Enigmail-Version:Content-Type; b=wTCYaNL6TpwAOm0mfzjmvwz70Fxzb9tlMcWn1Rawi7VMLs0QEXuqcb3C2KjEEJqlr pe3n8MS8wRXZAGyZckiRVYjejP7bD3NfWeTEJmhdSWiEKjxalfKW7+qOU0Ode3/wQ2 dtNkOBpUmEGpJWT3cX/hq4rmJnNSRNwYr2cMrP+w= Message-ID: <5023979B.4010903@yandex.ru> Date: Thu, 09 Aug 2012 14:57:31 +0400 From: "Andrey V. Elsukov" User-Agent: Mozilla Thunderbird 1.5 (FreeBSD/20051231) MIME-Version: 1.0 To: Lawrence Stewart References: <50237B73.9040301@freebsd.org> In-Reply-To: <50237B73.9040301@freebsd.org> X-Enigmail-Version: 1.4.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigCB4E32ED76DB1FA8FC647229" Cc: "freebsd-fs@freebsd.org" , Marcel Moolenaar Subject: Re: gpart rewrites pmbr in a way which breaks Win 7 EFI bootloader X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Aug 2012 10:57:44 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigCB4E32ED76DB1FA8FC647229 Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: quoted-printable On 09.08.2012 12:57, Lawrence Stewart wrote: > After identifying the cause of the problem and a workaround (please see= the README.txt at the above > URL for full details and pre/post gpart dumps of the MBR+GPT), I have t= he following questions: >=20 > - Should gpart be writing 0x80 (active) in the protective MBR entry? AFAIK, this was added because some BIOS could not boot without it. > - Why is Windows EFI bootloader so sensitive to 0x80 in pmbr? This question you should ask to the Microsoft. :) > - Should gpart be silently rewriting the protective MBR entry at all wh= en only asked to make changes > to the GPT? The PMBR is part of GPT metadata described in the UEFI spec. So, I think = it can. --=20 WBR, Andrey V. Elsukov --------------enigCB4E32ED76DB1FA8FC647229 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (MingW32) iQEcBAEBAgAGBQJQI5egAAoJEAHF6gQQyKF6QvgH/iPjNJAIYAUuEAa07+mN+TA5 flaGVh3IAYPX6azxkbntjQ3UHmSn9supba+6hLNfMrgNOYdmKdntfFL1WVJR8Tah lkmThWSV5IeLZM/L41G8k4bPe0cwGHhxupYqzG6pddMZamK8pNm2ERsZUBtSYidi L75ZvR3VTfl5lvRmKMzAZ5pKBjFsDBUVh3XMQ6HzZpGvIwH86t1mmZuspZSIr/Kf P65GZUFiNOOJ02xVjKIQR8ljJQM1BnMZ8K8qIurIvK+xaZVRn06EdgvGSJgDSZgX ibG8icV2apwON/trxst0cYXaOlP+CCEJr9H/SStJ8L1mtAN3NS6Tagez16sTEkM= =hjMX -----END PGP SIGNATURE----- --------------enigCB4E32ED76DB1FA8FC647229-- From owner-freebsd-fs@FreeBSD.ORG Thu Aug 9 12:11:53 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 724BE106566B; Thu, 9 Aug 2012 12:11:53 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id 316E48FC14; Thu, 9 Aug 2012 12:11:52 +0000 (UTC) Received: from lstewart1.loshell.room52.net (ppp59-167-184-191.static.internode.on.net [59.167.184.191]) by lauren.room52.net (Postfix) with ESMTPSA id 35AE47E896; Thu, 9 Aug 2012 22:11:51 +1000 (EST) Message-ID: <5023A907.7060800@freebsd.org> Date: Thu, 09 Aug 2012 22:11:51 +1000 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120613 Thunderbird/13.0 MIME-Version: 1.0 To: "Andrey V. Elsukov" References: <50237B73.9040301@freebsd.org> <5023979B.4010903@yandex.ru> In-Reply-To: <5023979B.4010903@yandex.ru> Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lauren.room52.net Cc: "freebsd-fs@freebsd.org" , Marcel Moolenaar Subject: Re: gpart rewrites pmbr in a way which breaks Win 7 EFI bootloader X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Aug 2012 12:11:53 -0000 Hi Andrey, On 08/09/12 20:57, Andrey V. Elsukov wrote: > On 09.08.2012 12:57, Lawrence Stewart wrote: >> After identifying the cause of the problem and a workaround (please see the README.txt at the above >> URL for full details and pre/post gpart dumps of the MBR+GPT), I have the following questions: >> >> - Should gpart be writing 0x80 (active) in the protective MBR entry? > > AFAIK, this was added because some BIOS could not boot without it. Makes sense if gpart is writing the pmbr out i.e. "gpart bootcode -b /boot/pmbr ", but manipulating an existing pmbr for a GPT specific subcommand smells dodgy to me. >> - Why is Windows EFI bootloader so sensitive to 0x80 in pmbr? > > This question you should ask to the Microsoft. :) Perhaps I should rephrase my question as: Is the MS bootloader's behaviour reasonable/unreasonable based on what people know of the relevant specs? My current guess why it behaves like this is that if it sees an MBR partition marked active, it simply assumes another OS is in charge and therefore bails out at the Windows EFI boot stage. >> - Should gpart be silently rewriting the protective MBR entry at all when only asked to make changes >> to the GPT? > > The PMBR is part of GPT metadata described in the UEFI spec. So, I think it can. Can and should are two different things. I would argue it's a POLA violation at the very least to manipulate the pmbr when the user asked for something else. I certainly started my hunting expecting to find the GPT changing in some subtle way when FreeBSD wrote it out compared to what Windows writes. We have a specific gpart command to put a pmbr in place so I think it's reasonable to expect other GPT specific commands not to fiddle with the pmbr. Cheers, Lawrence From owner-freebsd-fs@FreeBSD.ORG Thu Aug 9 14:20:29 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [69.147.83.53]) by hub.freebsd.org (Postfix) with ESMTP id EA4531065674; Thu, 9 Aug 2012 14:20:29 +0000 (UTC) (envelope-from ae@FreeBSD.org) Received: from [127.0.0.1] (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id ED1AC14E247; Thu, 9 Aug 2012 14:20:28 +0000 (UTC) Message-ID: <5023C723.90606@FreeBSD.org> Date: Thu, 09 Aug 2012 18:20:19 +0400 From: "Andrey V. Elsukov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:10.0.3) Gecko/20120406 Thunderbird/10.0.3 MIME-Version: 1.0 To: Lawrence Stewart References: <50237B73.9040301@freebsd.org> <5023979B.4010903@yandex.ru> <5023A907.7060800@freebsd.org> In-Reply-To: <5023A907.7060800@freebsd.org> X-Enigmail-Version: 1.4 OpenPGP: id=10C8A17A Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig88A52F6504CD8B59506F28AD" Cc: "freebsd-fs@freebsd.org" , Marcel Moolenaar Subject: Re: gpart rewrites pmbr in a way which breaks Win 7 EFI bootloader X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Aug 2012 14:20:30 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig88A52F6504CD8B59506F28AD Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: quoted-printable On 09.08.2012 16:11, Lawrence Stewart wrote: >>> - Should gpart be writing 0x80 (active) in the protective MBR entry? >> >> AFAIK, this was added because some BIOS could not boot without it. >=20 > Makes sense if gpart is writing the pmbr out i.e. "gpart bootcode -b > /boot/pmbr ", but manipulating an existing pmbr for a GPT specifi= c > subcommand smells dodgy to me. When you create GPT it already has PMBR, because it is part of GPT metadata. gpart's `bootcode' subcommand writes *bootcode* to specific area in the PMBR. >>> - Why is Windows EFI bootloader so sensitive to 0x80 in pmbr? >> >> This question you should ask to the Microsoft. :) >=20 > Perhaps I should rephrase my question as: >=20 > Is the MS bootloader's behaviour reasonable/unreasonable based on what > people know of the relevant specs? My current guess why it behaves like= > this is that if it sees an MBR partition marked active, it simply > assumes another OS is in charge and therefore bails out at the Windows > EFI boot stage. In the EFI system partition might be several boot loaders, and this MS bootloader's behaviour seems strange to me. > Can and should are two different things. I would argue it's a POLA > violation at the very least to manipulate the pmbr when the user asked > for something else. I certainly started my hunting expecting to find th= e > GPT changing in some subtle way when FreeBSD wrote it out compared to > what Windows writes. >=20 > We have a specific gpart command to put a pmbr in place so I think it's= > reasonable to expect other GPT specific commands not to fiddle with the= > pmbr. Don't confuse bootcode in the PMBR and PMBR. --=20 WBR, Andrey V. Elsukov --------------enig88A52F6504CD8B59506F28AD Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQEcBAEBAgAGBQJQI8coAAoJEAHF6gQQyKF6C9oH/1laFShQJp51rXIStTYZxQIc JFAqmXpMs1XjDbtSn6t3ZJuPAfW9NS+SzWE1cEPJ4dilUGZaYH37EbbCBtOdtCHt Z5yMgklFwClaRLmQd7gfxYdZwDmPBBtKVmi9K2xko+ES9RKLPdVyHewCnTHYYlh6 +U7Z9vOz7XFv/4pLoDWeogBE+hZFuai0i+6+tRKDXuMjzWiTeDaNadEqdG1psudL qdh09Kc0ysYWERmz/qogImC5c/BecGPX0R82uqC7Muj5lKauG86emjtdZlm0g3Vo KUqk6ojLhE85sDPZdIl+KWAG49RECCZcNtEq9uCNrsgQAOkPCOby0CF3NhWfdo4= =iSZ8 -----END PGP SIGNATURE----- --------------enig88A52F6504CD8B59506F28AD-- From owner-freebsd-fs@FreeBSD.ORG Thu Aug 9 14:39:02 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 80DD9106564A; Thu, 9 Aug 2012 14:39:02 +0000 (UTC) (envelope-from marcel@xcllnt.net) Received: from mail.xcllnt.net (mail.xcllnt.net [70.36.220.4]) by mx1.freebsd.org (Postfix) with ESMTP id 4D7A28FC16; Thu, 9 Aug 2012 14:39:02 +0000 (UTC) Received: from marcelm-sslvpn-nc.jnpr.net (natint3.juniper.net [66.129.224.36]) (authenticated bits=0) by mail.xcllnt.net (8.14.5/8.14.5) with ESMTP id q79Ecowj070803 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Thu, 9 Aug 2012 07:38:56 -0700 (PDT) (envelope-from marcel@xcllnt.net) Content-Type: text/plain; charset=koi8-r Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1485\)) From: Marcel Moolenaar In-Reply-To: <5023A907.7060800@freebsd.org> Date: Thu, 9 Aug 2012 07:38:45 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <50237B73.9040301@freebsd.org> <5023979B.4010903@yandex.ru> <5023A907.7060800@freebsd.org> To: Lawrence Stewart X-Mailer: Apple Mail (2.1485) Cc: "freebsd-fs@freebsd.org" , Marcel Moolenaar Subject: Re: gpart rewrites pmbr in a way which breaks Win 7 EFI bootloader X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Aug 2012 14:39:02 -0000 On Aug 9, 2012, at 5:11 AM, Lawrence Stewart = wrote: > Hi Andrey, >=20 > On 08/09/12 20:57, Andrey V. Elsukov wrote: >> On 09.08.2012 12:57, Lawrence Stewart wrote: >>> After identifying the cause of the problem and a workaround (please = see the README.txt at the above >>> URL for full details and pre/post gpart dumps of the MBR+GPT), I = have the following questions: >>>=20 >>> - Should gpart be writing 0x80 (active) in the protective MBR entry? >>=20 >> AFAIK, this was added because some BIOS could not boot without it. >=20 > Makes sense if gpart is writing the pmbr out i.e. "gpart bootcode -b > /boot/pmbr ", but manipulating an existing pmbr for a GPT = specific > subcommand smells dodgy to me. Agreed. The original design of gpart was such that it could preserve what it needed and only limit changes to what it was asked to do. For the GPT scheme this meant that it would simply read the entire PMBR, keep it in memory and write it back when updates are made. etc... >=20 >>> - Why is Windows EFI bootloader so sensitive to 0x80 in pmbr? >>=20 >> This question you should ask to the Microsoft. :) >=20 > Perhaps I should rephrase my question as: >=20 > Is the MS bootloader's behaviour reasonable/unreasonable based on what > people know of the relevant specs? My current guess why it behaves = like > this is that if it sees an MBR partition marked active, it simply > assumes another OS is in charge and therefore bails out at the Windows > EFI boot stage. I think the key distinction may be between BIOS and (U)EFI. When BIOS is the firmware, beaviour is different from when booting (U)EFI. It is possible (likely?) that Windows has different behaviour based on the firmware as well. >=20 >>> - Should gpart be silently rewriting the protective MBR entry at all = when only asked to make changes >>> to the GPT? >>=20 >> The PMBR is part of GPT metadata described in the UEFI spec. So, I = think it can. >=20 > Can and should are two different things. It should. The problem is not with geom_part writing the PMBR, it's with what we write and why. This loops back to the ol' compatibility discussions. > We have a specific gpart command to put a pmbr in place so I think = it's > reasonable to expect other GPT specific commands not to fiddle with = the > pmbr. I beg to differ. The PMBR is an integral part of the GPT spec, so it is not at all reasonable to expect the GPT scheme to bank on something or someone else to create or maintain it. What about the following: We have the kernel keep track of the firmware used. On x86 this is either BIOS or UEFI in the common case. Other F/W implementations like U-Boot, OFW, etc are possible as well, especially on non-x86 machines. The geom_part scheme uses this information to determine how to behave with respect to the PMBR. When booted with BIOS, non-standard stuff is accepted by virtue of what we've seen in the field. With UEFI we can start off being anal (read: strictly compliant) and extend out based on what we run into. In particular: this way we also don't mess up the EFI/GPT support that is there on ia64. Thoughts? --=20 Marcel Moolenaar marcel@xcllnt.net From owner-freebsd-fs@FreeBSD.ORG Thu Aug 9 16:07:53 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C3C141065673 for ; Thu, 9 Aug 2012 16:07:53 +0000 (UTC) (envelope-from daniel@digsys.bg) Received: from smtp-sofia.digsys.bg (smtp-sofia.digsys.bg [193.68.3.230]) by mx1.freebsd.org (Postfix) with ESMTP id 4C49C8FC14 for ; Thu, 9 Aug 2012 16:07:52 +0000 (UTC) Received: from dcave.digsys.bg (dcave.digsys.bg [192.92.129.5]) (authenticated bits=0) by smtp-sofia.digsys.bg (8.14.5/8.14.5) with ESMTP id q79FdZoA032835 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Thu, 9 Aug 2012 18:39:36 +0300 (EEST) (envelope-from daniel@digsys.bg) Message-ID: <5023D9B7.20001@digsys.bg> Date: Thu, 09 Aug 2012 18:39:35 +0300 From: Daniel Kalchev User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:10.0.6esrpre) Gecko/20120728 Thunderbird/10.0.6 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 7bit Subject: ZFS scrub CPU bound? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Aug 2012 16:07:53 -0000 I have built myself an server with SSD only drives. That thing goes "fast", like during scrub # zpool status pool: system state: ONLINE scan: scrub in progress since Thu Aug 9 18:30:46 2012 87.1G scanned out of 193G at 602M/s, 0h3m to go 0 repaired, 45.06% done config: NAME STATE READ WRITE CKSUM system ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 da0p3 ONLINE 0 0 0 da1p3 ONLINE 0 0 0 da2p3 ONLINE 0 0 0 da3p3 ONLINE 0 0 0 But, what I wonder is top -S # top -S last pid: 7912; load averages: 9.89, 3.26, 1.29 up 0+06:03:59 18:32:04 146 processes: 3 running, 142 sleeping, 1 waiting CPU: 0.4% user, 0.0% nice, 51.4% system, 0.8% interrupt, 47.3% idle Mem: 2171M Active, 1541M Inact, 5407M Wired, 25M Cache, 416K Buf, 22G Free Swap: 8192M Total, 8192M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 11 root 32 155 ki31 0K 512K CPU31 31 190.2H 1818.46% idle 0 root 268 -8 0 0K 4288K - 0 141:25 1261.33% kernel 4 root 4 -8 - 0K 80K CPU30 30 9:13 95.17% zfskern 13 root 3 -8 - 0K 48K - 4 5:37 42.97% geom 12 root 66 -84 - 0K 1056K WAIT 0 6:19 30.86% intr [...] It seems that zfskern will top to 100%. This is an 32 core system, and as you see scrub, at 600MB/sec is able to eat 16 cores (from 2x 2.2 GHz Opteron 6274). There is high load on geom as well... but geom does go over 100% CPU, so I suppose it scales. What might be there in ZFS code that is single-CPU bound? freebsd 9-stable. Daniel From owner-freebsd-fs@FreeBSD.ORG Thu Aug 9 23:11:48 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E5437106564A for ; Thu, 9 Aug 2012 23:11:48 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74]) by mx1.freebsd.org (Postfix) with ESMTP id 8153C8FC20 for ; Thu, 9 Aug 2012 23:11:48 +0000 (UTC) Received: from freddy.simplesystems.org (freddy.simplesystems.org [65.66.246.65]) by blade.simplesystems.org (8.14.4+Sun/8.14.4) with ESMTP id q79N0ujN003638; Thu, 9 Aug 2012 18:00:56 -0500 (CDT) Date: Thu, 9 Aug 2012 18:00:56 -0500 (CDT) From: Bob Friesenhahn X-X-Sender: bfriesen@freddy.simplesystems.org To: Daniel Kalchev In-Reply-To: <5023D9B7.20001@digsys.bg> Message-ID: References: <5023D9B7.20001@digsys.bg> User-Agent: Alpine 2.01 (GSO 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (blade.simplesystems.org [65.66.246.90]); Thu, 09 Aug 2012 18:00:56 -0500 (CDT) Cc: freebsd-fs@freebsd.org Subject: Re: ZFS scrub CPU bound? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Aug 2012 23:11:49 -0000 On Thu, 9 Aug 2012, Daniel Kalchev wrote: > > What might be there in ZFS code that is single-CPU bound? Almost certainly there is a single brain at the top to dictate the scrub algorithm. Scrub does need to proceed in appropriate order. Some things are best left for one CPU to do. It is unlikely that FreeBSD's I/O system is optimized for SSDs. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From owner-freebsd-fs@FreeBSD.ORG Fri Aug 10 01:14:13 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 548C4106564A; Fri, 10 Aug 2012 01:14:13 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id 0D7248FC0A; Fri, 10 Aug 2012 01:14:12 +0000 (UTC) Received: from lstewart1.loshell.room52.net (ppp59-167-184-191.static.internode.on.net [59.167.184.191]) by lauren.room52.net (Postfix) with ESMTPSA id 1C6DF7E820; Fri, 10 Aug 2012 11:14:11 +1000 (EST) Message-ID: <50246063.2080604@freebsd.org> Date: Fri, 10 Aug 2012 11:14:11 +1000 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120613 Thunderbird/13.0 MIME-Version: 1.0 To: "Andrey V. Elsukov" References: <50237B73.9040301@freebsd.org> <5023979B.4010903@yandex.ru> <5023A907.7060800@freebsd.org> <5023C723.90606@FreeBSD.org> In-Reply-To: <5023C723.90606@FreeBSD.org> Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lauren.room52.net Cc: "freebsd-fs@freebsd.org" , Marcel Moolenaar Subject: Re: gpart rewrites pmbr in a way which breaks Win 7 EFI bootloader X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Aug 2012 01:14:13 -0000 On 08/10/12 00:20, Andrey V. Elsukov wrote: > On 09.08.2012 16:11, Lawrence Stewart wrote: >>>> - Should gpart be writing 0x80 (active) in the protective MBR entry? >>> >>> AFAIK, this was added because some BIOS could not boot without it. >> >> Makes sense if gpart is writing the pmbr out i.e. "gpart bootcode -b >> /boot/pmbr ", but manipulating an existing pmbr for a GPT specific >> subcommand smells dodgy to me. > > When you create GPT it already has PMBR, because it is part of GPT > metadata. gpart's `bootcode' subcommand writes *bootcode* to specific > area in the PMBR. Right, I had a fundamental misunderstanding about the relationship between the pmbr and GPT. Thanks to you and Marcel for setting me straight on this. >>>> - Why is Windows EFI bootloader so sensitive to 0x80 in pmbr? >>> >>> This question you should ask to the Microsoft. :) >> >> Perhaps I should rephrase my question as: >> >> Is the MS bootloader's behaviour reasonable/unreasonable based on what >> people know of the relevant specs? My current guess why it behaves like >> this is that if it sees an MBR partition marked active, it simply >> assumes another OS is in charge and therefore bails out at the Windows >> EFI boot stage. > > In the EFI system partition might be several boot loaders, and this MS > bootloader's behaviour seems strange to me. > >> Can and should are two different things. I would argue it's a POLA >> violation at the very least to manipulate the pmbr when the user asked >> for something else. I certainly started my hunting expecting to find the >> GPT changing in some subtle way when FreeBSD wrote it out compared to >> what Windows writes. >> >> We have a specific gpart command to put a pmbr in place so I think it's >> reasonable to expect other GPT specific commands not to fiddle with the >> pmbr. > > Don't confuse bootcode in the PMBR and PMBR. Despite my misunderstanding about the relationship between the pmbr and GPT, I still stand by my assertion that gpart is not behaving very well here, but more on that in my follow up to Marcel. Cheers, Lawrence From owner-freebsd-fs@FreeBSD.ORG Fri Aug 10 02:13:02 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 63482106566B; Fri, 10 Aug 2012 02:13:02 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id D77EA8FC0A; Fri, 10 Aug 2012 02:13:01 +0000 (UTC) Received: from lstewart1.loshell.room52.net (ppp59-167-184-191.static.internode.on.net [59.167.184.191]) by lauren.room52.net (Postfix) with ESMTPSA id 7485F7E84A; Fri, 10 Aug 2012 12:13:00 +1000 (EST) Message-ID: <50246E2C.1070604@freebsd.org> Date: Fri, 10 Aug 2012 12:13:00 +1000 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120613 Thunderbird/13.0 MIME-Version: 1.0 To: Marcel Moolenaar References: <50237B73.9040301@freebsd.org> <5023979B.4010903@yandex.ru> <5023A907.7060800@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lauren.room52.net Cc: "freebsd-fs@freebsd.org" , Marcel Moolenaar Subject: Re: gpart rewrites pmbr in a way which breaks Win 7 EFI bootloader X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Aug 2012 02:13:02 -0000 [Note: all mail to Marcel is currently bouncing due to a dsbl entry for xcllnt.net's MX] On 08/10/12 00:38, Marcel Moolenaar wrote: > > On Aug 9, 2012, at 5:11 AM, Lawrence Stewart wrote: > >> Hi Andrey, >> >> On 08/09/12 20:57, Andrey V. Elsukov wrote: >>> On 09.08.2012 12:57, Lawrence Stewart wrote: >>>> After identifying the cause of the problem and a workaround (please see the README.txt at the above >>>> URL for full details and pre/post gpart dumps of the MBR+GPT), I have the following questions: >>>> >>>> - Should gpart be writing 0x80 (active) in the protective MBR entry? >>> >>> AFAIK, this was added because some BIOS could not boot without it. >> >> Makes sense if gpart is writing the pmbr out i.e. "gpart bootcode -b >> /boot/pmbr ", but manipulating an existing pmbr for a GPT specific >> subcommand smells dodgy to me. > > Agreed. The original design of gpart was such that it could preserve > what it needed and only limit changes to what it was asked to do. > > For the GPT scheme this meant that it would simply read the entire > PMBR, keep it in memory and write it back when updates are made. > > etc... That sounds like reasonable (and better) behaviour to me, but modulo Andrey's comments about some BIOS-based systems not booting without the pmbr partition being set to active, switching to this behaviour seems as though it may cause issues. >>>> - Why is Windows EFI bootloader so sensitive to 0x80 in pmbr? >>> >>> This question you should ask to the Microsoft. :) >> >> Perhaps I should rephrase my question as: >> >> Is the MS bootloader's behaviour reasonable/unreasonable based on what >> people know of the relevant specs? My current guess why it behaves like >> this is that if it sees an MBR partition marked active, it simply >> assumes another OS is in charge and therefore bails out at the Windows >> EFI boot stage. > > I think the key distinction may be between BIOS and (U)EFI. When BIOS > is the firmware, beaviour is different from when booting (U)EFI. It is > possible (likely?) that Windows has different behaviour based on the > firmware as well. On the HP system I'm seeing this on, it tries EFI boot sources first, then falls back to BIOS boot sources second. As a result of installing Win 7 from the DVD's EFI bootcode, which in turn sets up GPT and installs the Win 7 x64 EFI boot loader, Win 7 will only boot from the EFI boot loader on this system. After the pmbr entry gets the 0x80 active flag set by gpart, all subsequent reboots fall through from the EFI boot sources to the BIOS boot sources i.e. there is no way with this set up that Win is attempting to boot when the firmware switches to BIOS, as the pmbr has no boot code in it. Either the Win 7 EFI bootcode is being run and making a conscious choice to bail, or the HP EFI firmware is seeing the 0x80 in the pmbr and skipping running the Win 7 EFI bootcode. Not sure if there's a way I can tell which one is the case? >>>> - Should gpart be silently rewriting the protective MBR entry at all when only asked to make changes >>>> to the GPT? >>> >>> The PMBR is part of GPT metadata described in the UEFI spec. So, I think it can. >> >> Can and should are two different things. > > It should. The problem is not with geom_part writing the PMBR, it's > with what we write and why. This loops back to the ol' compatibility > discussions. Right, I didn't understand that the pmbr is actually part of the GPT metadata and therefore should always be written to disk as part of any GPT related gpart command. >> We have a specific gpart command to put a pmbr in place so I think it's >> reasonable to expect other GPT specific commands not to fiddle with the >> pmbr. > > I beg to differ. The PMBR is an integral part of the GPT spec, so it > is not at all reasonable to expect the GPT scheme to bank on something > or someone else to create or maintain it. > > What about the following: We have the kernel keep track of the firmware > used. On x86 this is either BIOS or UEFI in the common case. Other F/W > implementations like U-Boot, OFW, etc are possible as well, especially > on non-x86 machines. > > The geom_part scheme uses this information to determine how to behave > with respect to the PMBR. When booted with BIOS, non-standard stuff is > accepted by virtue of what we've seen in the field. With UEFI we can > start off being anal (read: strictly compliant) and extend out based > on what we run into. > > In particular: this way we also don't mess up the EFI/GPT support that > is there on ia64. > > Thoughts? I'm not very fluent in all this so I won't answer your question directly, but rather walk through a scenario and hopefully you can comment on how your proposal works in this case... Taking my desired HP configuration as the example: - Win 7 installed as an EFI boot source - FreeBSD doesn't have EFI boot support, so it has to be installed to boot via pmbr bootcode + gpt(zfs)boot in a freebsd-boot GPT partition - In the HP's firmware config, I choose whether to try boot from EFI or "legacy" (BIOS) sources first. Currently I try from EFI first. - In this arrangement, the firmware starts in EFI mode and will always boot Win 7 as it's the only bootable EFI payload installed. - Using the F9 key on boot, I get a menu to choose the boot source, and with FreeBSD installed to boot via pmbr bootcode + gpt(zfs)boot, the only way to get FreeBSD to start is to select "Legacy HDD" as the boot source, which switches firmware to BIOS and boots from the bootcode in the pmbr, which in turn executes gpt(zfs)boot from the freebsd-boot GPT partition. As I understand things, the above scenario ensures FreeBSD always thinks the firmware is BIOS even though an EFI bootloader is installed and functional. Therefore, your proposed tweaks don't handle this case as gpart will think it's running on a BIOS systems and happily tweak the pmbr in ways which can (and in this case do) break the EFI booting. The only way I can see to get smart about this would be to actually look for the existence of an EFI system partition and look to see if there is an active bootloader payload in there. Sounds pretty gross to me but I can't think of a way around this. This of course all gets greatly simplified when FreeBSD grows EFI boot support. However in the meantime, I see the above as the only viable solution to dual boot Win 7 EFI and FreeBSD 9 on the same disk (please suggest alternatives if I'm missing something obvious). Cheers, Lawrence From owner-freebsd-fs@FreeBSD.ORG Fri Aug 10 08:31:58 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3116D106566C; Fri, 10 Aug 2012 08:31:58 +0000 (UTC) (envelope-from ae@FreeBSD.org) Received: from mail.kirov.so-ups.ru (mail.kirov.so-ups.ru [178.74.170.1]) by mx1.freebsd.org (Postfix) with ESMTP id C60108FC15; Fri, 10 Aug 2012 08:31:57 +0000 (UTC) Received: from kas30pipe.localhost (localhost.kirov.so-ups.ru [127.0.0.1]) by mail.kirov.so-ups.ru (Postfix) with SMTP id EF8E2B8027; Fri, 10 Aug 2012 12:31:45 +0400 (MSK) Received: from kirov.so-ups.ru (unknown [172.21.81.1]) by mail.kirov.so-ups.ru (Postfix) with ESMTP id E9C18B8024; Fri, 10 Aug 2012 12:31:45 +0400 (MSK) Received: by ns.kirov.so-ups.ru (Postfix, from userid 1010) id E412BBA09E; Fri, 10 Aug 2012 12:31:45 +0400 (MSK) Received: from [127.0.0.1] (elsukov.kirov.oduur.so [10.118.3.52]) by ns.kirov.so-ups.ru (Postfix) with ESMTP id AEA8BBA09A; Fri, 10 Aug 2012 12:31:45 +0400 (MSK) Message-ID: <5024C6EB.1040109@FreeBSD.org> Date: Fri, 10 Aug 2012 12:31:39 +0400 From: "Andrey V. Elsukov" User-Agent: Mozilla Thunderbird 1.5 (FreeBSD/20051231) MIME-Version: 1.0 To: Lawrence Stewart References: <50237B73.9040301@freebsd.org> <5023979B.4010903@yandex.ru> <5023A907.7060800@freebsd.org> <50246E2C.1070604@freebsd.org> In-Reply-To: <50246E2C.1070604@freebsd.org> X-Enigmail-Version: 1.4.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig6C303CBB45C369C8A550008C" X-SpamTest-Version: SMTP-Filter Version 3.0.0 [0284], KAS30/Release X-SpamTest-Info: Not protected Cc: "freebsd-fs@freebsd.org" , Marcel Moolenaar Subject: Re: gpart rewrites pmbr in a way which breaks Win 7 EFI bootloader X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Aug 2012 08:31:58 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig6C303CBB45C369C8A550008C Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: quoted-printable On 10.08.2012 6:13, Lawrence Stewart wrote: >> What about the following: We have the kernel keep track of the firmwar= e >> used. On x86 this is either BIOS or UEFI in the common case. Other F/W= >> implementations like U-Boot, OFW, etc are possible as well, especially= >> on non-x86 machines. >> >> The geom_part scheme uses this information to determine how to behave >> with respect to the PMBR. When booted with BIOS, non-standard stuff is= >> accepted by virtue of what we've seen in the field. With UEFI we can >> start off being anal (read: strictly compliant) and extend out based >> on what we run into. >> >> In particular: this way we also don't mess up the EFI/GPT support that= >> is there on ia64. >> >> Thoughts? It seems this is not enough. The problem, that Lawrence has, is not related to BIOS/UEFI. Automatic detection of various standard violation i= s handy for our users, but sometimes false positives occur. To solve all problems it seems we need to introduce several quirks, e.g.:= #define GPT_QUIRK_BOOTCAMP 0x0001 /* boot camp supported */ #define GPT_QUIRK_MSLOADER 0x0002 /* don't set Active flag to the P= MBR entry */ #define GPT_QUIRK_IGNOREPMBR 0x0004 /* don't require PMBR entry */ The same quirks should be added to the loader(8). --=20 WBR, Andrey V. Elsukov --------------enig6C303CBB45C369C8A550008C Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (MingW32) iQEcBAEBAgAGBQJQJMbxAAoJEAHF6gQQyKF69JMIAJtcBy9l2eLRvPhJ1YMW/SBP 7hWXYHodzAyrBUCvRzTgA3x7wbHfsCsR0qDi7CKUyfPN/4WQlP8l0SzLgssV357r z3Ly3/7wiOaCLV0Kw70/J7h0zay2IO8ab/WioVfckm/13K7/hoUxw6FSNbMfkUFT 68acW67W4XzEPInh9PndtnAjv4cGg9k1ShrUaLa22XJC2OjGMDGY2p33AXY+zoWM vdq7VxE2dNciCt/uGdA+eQw9PvZLrvh4j8i/PJCa9VZxNaLimoSLQxeCSMw7Gr5h vutnkSH1axhFiFWVBUkk1Rn8MEoYuz72P/8seqypGCEscDDTqzg/otF0y5wTq58= =1aut -----END PGP SIGNATURE----- --------------enig6C303CBB45C369C8A550008C-- From owner-freebsd-fs@FreeBSD.ORG Fri Aug 10 10:14:17 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9B596106564A; Fri, 10 Aug 2012 10:14:17 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id 56EC98FC08; Fri, 10 Aug 2012 10:14:17 +0000 (UTC) Received: from [172.16.7.56] (ppp59-167-184-191.static.internode.on.net [59.167.184.191]) by lauren.room52.net (Postfix) with ESMTPSA id C52A17E820; Fri, 10 Aug 2012 20:14:09 +1000 (EST) Message-ID: <5024DEF1.3050900@freebsd.org> Date: Fri, 10 Aug 2012 20:14:09 +1000 From: Lawrence Stewart User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:14.0) Gecko/20120713 Thunderbird/14.0 MIME-Version: 1.0 To: Marcel Moolenaar References: <50237B73.9040301@freebsd.org> <5023979B.4010903@yandex.ru> <5023A907.7060800@freebsd.org> <50246E2C.1070604@freebsd.org> In-Reply-To: <50246E2C.1070604@freebsd.org> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lauren.room52.net Cc: "freebsd-fs@freebsd.org" , ae@freebsd.org, Marcel Moolenaar Subject: Re: gpart rewrites pmbr in a way which breaks Win 7 EFI bootloader X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Aug 2012 10:14:17 -0000 On 10/08/2012 12:13 PM, Lawrence Stewart wrote: [snip] > Taking my desired HP configuration as the example: > > - Win 7 installed as an EFI boot source > > - FreeBSD doesn't have EFI boot support, so it has to be installed to > boot via pmbr bootcode + gpt(zfs)boot in a freebsd-boot GPT partition > > - In the HP's firmware config, I choose whether to try boot from EFI or > "legacy" (BIOS) sources first. Currently I try from EFI first. > > - In this arrangement, the firmware starts in EFI mode and will always > boot Win 7 as it's the only bootable EFI payload installed. > > - Using the F9 key on boot, I get a menu to choose the boot source, and > with FreeBSD installed to boot via pmbr bootcode + gpt(zfs)boot, the > only way to get FreeBSD to start is to select "Legacy HDD" as the boot > source, which switches firmware to BIOS and boots from the bootcode in > the pmbr, which in turn executes gpt(zfs)boot from the freebsd-boot GPT > partition. I just wanted to confirm for posterity's sake that as of a few hours ago, the above set up does actually work. I have Win 7 installed using GPT + EFI boot. I booted into a FreeBSD 9.1 CD live filesystem and used "gpart -b /boot/pmbr -p /boot/gptzfsboot -i 6 raid/r0" to install first stage boot code into the pmbr and second stage boot code into my freebsd-boot GPT partition. I installed a ZFS-based FreeBSD set up into it's own GPT partition. I then tweaked the pmbr using my dd + vim trick to unset the active flag. I can now boot successfully from Win 7 EFI bootloader or, if I select "Legacy HDD" from my HP's boot menu, it will happily boot my FreeBSD install from the kernel living in my ZFS root. It's good to confirm this actually works as I expected it would, but I guess we need to make some changes in order to address this issue completely, including all the edge cases. Cheers, Lawrence From owner-freebsd-fs@FreeBSD.ORG Fri Aug 10 10:45:55 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1B8B91065672; Fri, 10 Aug 2012 10:45:55 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id CB82B8FC18; Fri, 10 Aug 2012 10:45:54 +0000 (UTC) Received: from [172.16.7.56] (ppp59-167-184-191.static.internode.on.net [59.167.184.191]) by lauren.room52.net (Postfix) with ESMTPSA id 7577F7E820; Fri, 10 Aug 2012 20:45:53 +1000 (EST) Message-ID: <5024E661.6080202@freebsd.org> Date: Fri, 10 Aug 2012 20:45:53 +1000 From: Lawrence Stewart User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:14.0) Gecko/20120713 Thunderbird/14.0 MIME-Version: 1.0 To: "Andrey V. Elsukov" References: <50237B73.9040301@freebsd.org> <5023979B.4010903@yandex.ru> <5023A907.7060800@freebsd.org> <50246E2C.1070604@freebsd.org> <5024C6EB.1040109@FreeBSD.org> In-Reply-To: <5024C6EB.1040109@FreeBSD.org> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lauren.room52.net Cc: "freebsd-fs@freebsd.org" , Marcel Moolenaar Subject: Re: gpart rewrites pmbr in a way which breaks Win 7 EFI bootloader X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Aug 2012 10:45:55 -0000 On 10/08/2012 6:31 PM, Andrey V. Elsukov wrote: > On 10.08.2012 6:13, Lawrence Stewart wrote: >>> What about the following: We have the kernel keep track of the firmware >>> used. On x86 this is either BIOS or UEFI in the common case. Other F/W >>> implementations like U-Boot, OFW, etc are possible as well, especially >>> on non-x86 machines. >>> >>> The geom_part scheme uses this information to determine how to behave >>> with respect to the PMBR. When booted with BIOS, non-standard stuff is >>> accepted by virtue of what we've seen in the field. With UEFI we can >>> start off being anal (read: strictly compliant) and extend out based >>> on what we run into. >>> >>> In particular: this way we also don't mess up the EFI/GPT support that >>> is there on ia64. >>> >>> Thoughts? > > It seems this is not enough. The problem, that Lawrence has, is not > related to BIOS/UEFI. Automatic detection of various standard violation is > handy for our users, but sometimes false positives occur. > > To solve all problems it seems we need to introduce several quirks, e.g.: > > #define GPT_QUIRK_BOOTCAMP 0x0001 /* boot camp supported */ > #define GPT_QUIRK_MSLOADER 0x0002 /* don't set Active flag to the PMBR entry */ > #define GPT_QUIRK_IGNOREPMBR 0x0004 /* don't require PMBR entry */ > > The same quirks should be added to the loader(8). This sounds like a useful proposal. I will note though that I can't be totally sure if my issue is caused by the MS EFI loader or my HP EFI firmware seeing the active flag and failing to boot. My hunch is it's the MS EFI loader because the HP EFI firmware still shows the MS EFI payload as a valid boot option and seems to switch to it temporarily before falling through to legacy BIOS boot. There are no diagnostic messages or anything shown on screen though so that makes it hard to know what's going on. Cheers, Lawrence