From owner-freebsd-fs@FreeBSD.ORG Mon Sep 10 00:18:15 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id ADBDA1065686 for ; Mon, 10 Sep 2012 00:18:15 +0000 (UTC) (envelope-from nicolas@i.0x5.de) Received: from n.0x5.de (n.0x5.de [217.197.85.144]) by mx1.freebsd.org (Postfix) with ESMTP id 674688FC17 for ; Mon, 10 Sep 2012 00:18:14 +0000 (UTC) Received: by pc5.i.0x5.de (Postfix, from userid 1003) id 3XFVG81fVxz7yTY; Mon, 10 Sep 2012 02:18:08 +0200 (CEST) Date: Mon, 10 Sep 2012 02:18:08 +0200 From: Nicolas Rachinsky To: freebsd-fs@freebsd.org Message-ID: <20120910001808.GA96418@mid.pc5.i.0x5.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Powered-by: FreeBSD X-Homepage: http://www.rachinsky.de X-PGP-Keyid: 887BAE72 X-PGP-Fingerprint: 039E 9433 115F BC5F F88D 4524 5092 45C4 887B AE72 X-PGP-Keys: http://www.rachinsky.de/nicolas/gpg/nicolas_rachinsky.asc User-Agent: Mutt/1.5.21 (2010-09-15) Subject: zfs, rsync and broken symlinks (cannot get extended attributes) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Sep 2012 00:18:15 -0000 Hallo, I tried to setup a backup with rsync and --fake-super. This does work fine, until I export and import the zpool (or reboot). After that I have symlinks that cannot have extended attributes: root@paladin /pool1/TST/etc/rc0.d# lsextattr -h user * K01atd rsync.%stat K01exim4 rsync.%stat K01heartbeat rsync.%stat K01logd rsync.%stat K01mysql-proxy rsync.%stat lsextattr: K01urandom: failed: Not a directory I was unable to reproduce this without using rsync. It seems to work fine if I only rsync /etc/rc0.d, but it fails if I rsync the whole /etc/rc0.d. Am I doing something wrong? This is on FreeBSD 8.3-RELEASE-p3, rsync 3.0.9 and the source is some debian squeeze. The zpool was recreated just before this tests (raidz1 with 4 disks). The system is on ufs. If there is anything missing, just ask. At the moment I can try some things with this system. Thanks in advance. Nicolas -- http://www.rachinsky.de/nicolas From owner-freebsd-fs@FreeBSD.ORG Mon Sep 10 10:30:10 2012 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0DA5C106564A for ; Mon, 10 Sep 2012 10:30:10 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D33D68FC17 for ; Mon, 10 Sep 2012 10:30:09 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q8AAU997087307 for ; Mon, 10 Sep 2012 10:30:09 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q8AAU9mu087291; Mon, 10 Sep 2012 10:30:09 GMT (envelope-from gnats) Date: Mon, 10 Sep 2012 10:30:09 GMT Message-Id: <201209101030.q8AAU9mu087291@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Martin Matuska Cc: Subject: Re: kern/156781: [zfs] zfs is losing the snapshot directory, X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Martin Matuska List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Sep 2012 10:30:10 -0000 The following reply was made to PR kern/156781; it has been noted by GNATS. From: Martin Matuska To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/156781: [zfs] zfs is losing the snapshot directory, Date: Mon, 10 Sep 2012 12:25:49 +0200 As I am not experiencing this problem on my systems, it would be great to create a way to reproduce it. E.g. a script that creates/deletes 1000 snapshots, mounts/unmounts them, etc. -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-freebsd-fs@FreeBSD.ORG Mon Sep 10 11:09:36 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CC6A010656AB for ; Mon, 10 Sep 2012 11:09:36 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id B587A8FC14 for ; Mon, 10 Sep 2012 11:09:36 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q8AB9aZs065268 for ; Mon, 10 Sep 2012 11:09:36 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q8AB9Y82064878 for freebsd-fs@FreeBSD.org; Mon, 10 Sep 2012 11:09:34 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 10 Sep 2012 11:09:34 GMT Message-Id: <201209101109.q8AB9Y82064878@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Sep 2012 11:09:36 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/171415 fs [zfs] zfs recv fails with "cannot receive incremental o kern/170945 fs [gpt] disk layout not portable between direct connect o kern/170914 fs [zfs] [patch] Import patchs related with issues 3090 a o kern/170912 fs [zfs] [patch] unnecessarily setting DS_FLAG_INCONSISTE o bin/170778 fs [zfs] [panic] FreeBSD panics randomly o kern/170680 fs [nfs] Multiple NFS Client bug in the FreeBSD 7.4-RELEA o kern/170497 fs [xfs][panic] kernel will panic whenever I ls a mounted o kern/170238 fs [zfs] [panic] Panic when deleting data o kern/169945 fs [zfs] [panic] Kernel panic while importing zpool (afte o kern/169480 fs [zfs] ZFS stalls on heavy I/O o kern/169398 fs [zfs] Can't remove file with permanent error o kern/169339 fs panic while " : > /etc/123" o kern/169319 fs [zfs] zfs resilver can't complete o kern/168947 fs [nfs] [zfs] .zfs/snapshot directory is messed up when o kern/168942 fs [nfs] [hang] nfsd hangs after being restarted (not -HU o kern/168158 fs [zfs] incorrect parsing of sharenfs options in zfs (fs o kern/167979 fs [ufs] DIOCGDINFO ioctl does not work on 8.2 file syste o kern/167977 fs [smbfs] mount_smbfs results are differ when utf-8 or U o kern/167688 fs [fusefs] Incorrect signal handling with direct_io o kern/167685 fs [zfs] ZFS on USB drive prevents shutdown / reboot o kern/167612 fs [portalfs] The portal file system gets stuck inside po o kern/167272 fs [zfs] ZFS Disks reordering causes ZFS to pick the wron o kern/167260 fs [msdosfs] msdosfs disk was mounted the second time whe o kern/167109 fs [zfs] [panic] zfs diff kernel panic Fatal trap 9: gene o kern/167105 fs [nfs] mount_nfs can not handle source exports wiht mor o kern/167067 fs [zfs] [panic] ZFS panics the server o kern/167066 fs [zfs] ZVOLs not appearing in /dev/zvol o kern/167065 fs [zfs] boot fails when a spare is the boot disk o kern/167048 fs [nfs] [patch] RELEASE-9 crash when using ZFS+NULLFS+NF o kern/166912 fs [ufs] [panic] Panic after converting Softupdates to jo o kern/166851 fs [zfs] [hang] Copying directory from the mounted UFS di o kern/166477 fs [nfs] NFS data corruption. o kern/165950 fs [ffs] SU+J and fsck problem o kern/165923 fs [nfs] Writing to NFS-backed mmapped files fails if flu o kern/165521 fs [zfs] [hang] livelock on 1 Gig of RAM with zfs when 31 o kern/165392 fs Multiple mkdir/rmdir fails with errno 31 o kern/165087 fs [unionfs] lock violation in unionfs o kern/164472 fs [ufs] fsck -B panics on particular data inconsistency o kern/164370 fs [zfs] zfs destroy for snapshot fails on i386 and sparc o kern/164261 fs [nullfs] [patch] fix panic with NFS served from NULLFS o kern/164256 fs [zfs] device entry for volume is not created after zfs o kern/164184 fs [ufs] [panic] Kernel panic with ufs_makeinode o kern/163801 fs [md] [request] allow mfsBSD legacy installed in 'swap' o kern/163770 fs [zfs] [hang] LOR between zfs&syncer + vnlru leading to o kern/163501 fs [nfs] NFS exporting a dir and a subdir in that dir to o kern/162944 fs [coda] Coda file system module looks broken in 9.0 o kern/162860 fs [zfs] Cannot share ZFS filesystem to hosts with a hyph o kern/162751 fs [zfs] [panic] kernel panics during file operations o kern/162591 fs [nullfs] cross-filesystem nullfs does not work as expe o kern/162519 fs [zfs] "zpool import" relies on buggy realpath() behavi o kern/162362 fs [snapshots] [panic] ufs with snapshot(s) panics when g o kern/161968 fs [zfs] [hang] renaming snapshot with -r including a zvo p kern/161897 fs [zfs] [patch] zfs partition probing causing long delay o kern/161864 fs [ufs] removing journaling from UFS partition fails on o bin/161807 fs [patch] add option for explicitly specifying metadata o kern/161579 fs [smbfs] FreeBSD sometimes panics when an smb share is o kern/161533 fs [zfs] [panic] zfs receive panic: system ioctl returnin o kern/161438 fs [zfs] [panic] recursed on non-recursive spa_namespace_ o kern/161424 fs [nullfs] __getcwd() calls fail when used on nullfs mou o kern/161280 fs [zfs] Stack overflow in gptzfsboot o kern/161205 fs [nfs] [pfsync] [regression] [build] Bug report freebsd o kern/161169 fs [zfs] [panic] ZFS causes kernel panic in dbuf_dirty o kern/161112 fs [ufs] [lor] filesystem LOR in FreeBSD 9.0-BETA3 o kern/160893 fs [zfs] [panic] 9.0-BETA2 kernel panic o kern/160860 fs [ufs] Random UFS root filesystem corruption with SU+J o kern/160801 fs [zfs] zfsboot on 8.2-RELEASE fails to boot from root-o o kern/160790 fs [fusefs] [panic] VPUTX: negative ref count with FUSE o kern/160777 fs [zfs] [hang] RAID-Z3 causes fatal hang upon scrub/impo o kern/160706 fs [zfs] zfs bootloader fails when a non-root vdev exists o kern/160591 fs [zfs] Fail to boot on zfs root with degraded raidz2 [r o kern/160410 fs [smbfs] [hang] smbfs hangs when transferring large fil o kern/160283 fs [zfs] [patch] 'zfs list' does abort in make_dataset_ha o kern/159930 fs [ufs] [panic] kernel core o kern/159402 fs [zfs][loader] symlinks cause I/O errors o kern/159357 fs [zfs] ZFS MAXNAMELEN macro has confusing name (off-by- o kern/159356 fs [zfs] [patch] ZFS NAME_ERR_DISKLIKE check is Solaris-s o kern/159351 fs [nfs] [patch] - divide by zero in mountnfs() o kern/159251 fs [zfs] [request]: add FLETCHER4 as DEDUP hash option o kern/159077 fs [zfs] Can't cd .. with latest zfs version o kern/159048 fs [smbfs] smb mount corrupts large files o kern/159045 fs [zfs] [hang] ZFS scrub freezes system o kern/158839 fs [zfs] ZFS Bootloader Fails if there is a Dead Disk o kern/158802 fs amd(8) ICMP storm and unkillable process. o kern/158231 fs [nullfs] panic on unmounting nullfs mounted over ufs o f kern/157929 fs [nfs] NFS slow read o kern/157399 fs [zfs] trouble with: mdconfig force delete && zfs strip o kern/157179 fs [zfs] zfs/dbuf.c: panic: solaris assert: arc_buf_remov o kern/156797 fs [zfs] [panic] Double panic with FreeBSD 9-CURRENT and o kern/156781 fs [zfs] zfs is losing the snapshot directory, p kern/156545 fs [ufs] mv could break UFS on SMP systems o kern/156193 fs [ufs] [hang] UFS snapshot hangs && deadlocks processes o kern/156039 fs [nullfs] [unionfs] nullfs + unionfs do not compose, re o kern/155615 fs [zfs] zfs v28 broken on sparc64 -current o kern/155587 fs [zfs] [panic] kernel panic with zfs p kern/155411 fs [regression] [8.2-release] [tmpfs]: mount: tmpfs : No o kern/155199 fs [ext2fs] ext3fs mounted as ext2fs gives I/O errors o bin/155104 fs [zfs][patch] use /dev prefix by default when importing o kern/154930 fs [zfs] cannot delete/unlink file from full volume -> EN o kern/154828 fs [msdosfs] Unable to create directories on external USB o kern/154491 fs [smbfs] smb_co_lock: recursive lock for object 1 p kern/154228 fs [md] md getting stuck in wdrain state o kern/153996 fs [zfs] zfs root mount error while kernel is not located o kern/153753 fs [zfs] ZFS v15 - grammatical error when attempting to u o kern/153716 fs [zfs] zpool scrub time remaining is incorrect o kern/153695 fs [patch] [zfs] Booting from zpool created on 4k-sector o kern/153680 fs [xfs] 8.1 failing to mount XFS partitions o kern/153520 fs [zfs] Boot from GPT ZFS root on HP BL460c G1 unstable o kern/153418 fs [zfs] [panic] Kernel Panic occurred writing to zfs vol o kern/153351 fs [zfs] locking directories/files in ZFS o bin/153258 fs [patch][zfs] creating ZVOLs requires `refreservation' s kern/153173 fs [zfs] booting from a gzip-compressed dataset doesn't w o bin/153142 fs [zfs] ls -l outputs `ls: ./.zfs: Operation not support o kern/153126 fs [zfs] vdev failure, zpool=peegel type=vdev.too_small o kern/152022 fs [nfs] nfs service hangs with linux client [regression] o kern/151942 fs [zfs] panic during ls(1) zfs snapshot directory o kern/151905 fs [zfs] page fault under load in /sbin/zfs o bin/151713 fs [patch] Bug in growfs(8) with respect to 32-bit overfl o kern/151648 fs [zfs] disk wait bug o kern/151629 fs [fs] [patch] Skip empty directory entries during name o kern/151330 fs [zfs] will unshare all zfs filesystem after execute a o kern/151326 fs [nfs] nfs exports fail if netgroups contain duplicate o kern/151251 fs [ufs] Can not create files on filesystem with heavy us o kern/151226 fs [zfs] can't delete zfs snapshot o kern/151111 fs [zfs] vnodes leakage during zfs unmount o kern/150503 fs [zfs] ZFS disks are UNAVAIL and corrupted after reboot o kern/150501 fs [zfs] ZFS vdev failure vdev.bad_label on amd64 o kern/150390 fs [zfs] zfs deadlock when arcmsr reports drive faulted o kern/150336 fs [nfs] mountd/nfsd became confused; refused to reload n o kern/149208 fs mksnap_ffs(8) hang/deadlock o kern/149173 fs [patch] [zfs] make OpenSolaris installa o kern/149015 fs [zfs] [patch] misc fixes for ZFS code to build on Glib o kern/149014 fs [zfs] [patch] declarations in ZFS libraries/utilities o kern/149013 fs [zfs] [patch] make ZFS makefiles use the libraries fro o kern/148504 fs [zfs] ZFS' zpool does not allow replacing drives to be o kern/148490 fs [zfs]: zpool attach - resilver bidirectionally, and re o kern/148368 fs [zfs] ZFS hanging forever on 8.1-PRERELEASE o kern/148138 fs [zfs] zfs raidz pool commands freeze o kern/147903 fs [zfs] [panic] Kernel panics on faulty zfs device o kern/147881 fs [zfs] [patch] ZFS "sharenfs" doesn't allow different " p kern/147560 fs [zfs] [boot] Booting 8.1-PRERELEASE raidz system take o kern/147420 fs [ufs] [panic] ufs_dirbad, nullfs, jail panic (corrupt o kern/146941 fs [zfs] [panic] Kernel Double Fault - Happens constantly o kern/146786 fs [zfs] zpool import hangs with checksum errors o kern/146708 fs [ufs] [panic] Kernel panic in softdep_disk_write_compl o kern/146528 fs [zfs] Severe memory leak in ZFS on i386 o kern/146502 fs [nfs] FreeBSD 8 NFS Client Connection to Server s kern/145712 fs [zfs] cannot offline two drives in a raidz2 configurat o kern/145411 fs [xfs] [panic] Kernel panics shortly after mounting an f bin/145309 fs bsdlabel: Editing disk label invalidates the whole dev o kern/145272 fs [zfs] [panic] Panic during boot when accessing zfs on o kern/145246 fs [ufs] dirhash in 7.3 gratuitously frees hashes when it o kern/145238 fs [zfs] [panic] kernel panic on zpool clear tank o kern/145229 fs [zfs] Vast differences in ZFS ARC behavior between 8.0 o kern/145189 fs [nfs] nfsd performs abysmally under load o kern/144929 fs [ufs] [lor] vfs_bio.c + ufs_dirhash.c p kern/144447 fs [zfs] sharenfs fsunshare() & fsshare_main() non functi o kern/144416 fs [panic] Kernel panic on online filesystem optimization s kern/144415 fs [zfs] [panic] kernel panics on boot after zfs crash o kern/144234 fs [zfs] Cannot boot machine with recent gptzfsboot code o kern/143825 fs [nfs] [panic] Kernel panic on NFS client o bin/143572 fs [zfs] zpool(1): [patch] The verbose output from iostat o kern/143212 fs [nfs] NFSv4 client strange work ... o kern/143184 fs [zfs] [lor] zfs/bufwait LOR o kern/142878 fs [zfs] [vfs] lock order reversal o kern/142597 fs [ext2fs] ext2fs does not work on filesystems with real o kern/142489 fs [zfs] [lor] allproc/zfs LOR o kern/142466 fs Update 7.2 -> 8.0 on Raid 1 ends with screwed raid [re o kern/142306 fs [zfs] [panic] ZFS drive (from OSX Leopard) causes two o kern/142068 fs [ufs] BSD labels are got deleted spontaneously o kern/141897 fs [msdosfs] [panic] Kernel panic. msdofs: file name leng o kern/141463 fs [nfs] [panic] Frequent kernel panics after upgrade fro o kern/141305 fs [zfs] FreeBSD ZFS+sendfile severe performance issues ( o kern/141091 fs [patch] [nullfs] fix panics with DIAGNOSTIC enabled o kern/141086 fs [nfs] [panic] panic("nfs: bioread, not dir") on FreeBS o kern/141010 fs [zfs] "zfs scrub" fails when backed by files in UFS2 o kern/140888 fs [zfs] boot fail from zfs root while the pool resilveri o kern/140661 fs [zfs] [patch] /boot/loader fails to work on a GPT/ZFS- o kern/140640 fs [zfs] snapshot crash o kern/140068 fs [smbfs] [patch] smbfs does not allow semicolon in file o kern/139725 fs [zfs] zdb(1) dumps core on i386 when examining zpool c o kern/139715 fs [zfs] vfs.numvnodes leak on busy zfs p bin/139651 fs [nfs] mount(8): read-only remount of NFS volume does n o kern/139407 fs [smbfs] [panic] smb mount causes system crash if remot o kern/138662 fs [panic] ffs_blkfree: freeing free block o kern/138421 fs [ufs] [patch] remove UFS label limitations o kern/138202 fs mount_msdosfs(1) see only 2Gb o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136873 fs [ntfs] Missing directories/files on NTFS volume o kern/136865 fs [nfs] [patch] NFS exports atomic and on-the-fly atomic p kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes o kern/131441 fs [unionfs] [nullfs] unionfs and/or nullfs not combineab o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/127787 fs [lor] [ufs] Three LORs: vfslock/devfs/vfslock, ufs/vfs o bin/127270 fs fsck_msdosfs(8) may crash if BytesPerSec is zero o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file o kern/125895 fs [ffs] [panic] kernel: panic: ffs_blkfree: freeing free s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS o kern/123939 fs [msdosfs] corrupts new files o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F o kern/118912 fs [2tb] disk sizing/geometry problem with large array o kern/118713 fs [minidump] [patch] Display media size required for a k o kern/118318 fs [nfs] NFS server hangs under special circumstances o bin/118249 fs [ufs] mv(1): moving a directory changes its mtime o kern/118126 fs [nfs] [patch] Poor NFS server write performance o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o kern/117954 fs [ufs] dirhash on very large directories blocks the mac o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o conf/116931 fs lack of fsck_cd9660 prevents mounting iso images with o kern/116583 fs [ffs] [hang] System freezes for short time when using o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106107 fs [ufs] left-over fsck_snapshot after unfinished backgro o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes s bin/97498 fs [request] newfs(8) has no option to clear the first 12 o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [cd9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o bin/94810 fs fsck(8) incorrectly reports 'file system marked clean' o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88555 fs [panic] ffs_blkfree: freeing free frag on AMD 64 o kern/88266 fs [smbfs] smbfs does not implement UIO_NOCOPY and sendfi o bin/87966 fs [patch] newfs(8): introduce -A flag for newfs to enabl o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o bin/85494 fs fsck_ffs: unchecked use of cg_inosused macro etc. o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o bin/74779 fs Background-fsck checks one filesystem twice and omits o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o bin/70600 fs fsck(8) throws files away when it can't grow lost+foun o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o bin/27687 fs fsck(8) wrapper is not properly passing options to fsc o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 289 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Sep 10 11:50:08 2012 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E97991065677 for ; Mon, 10 Sep 2012 11:50:07 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 601F48FC1B for ; Mon, 10 Sep 2012 11:50:07 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q8ABo7wZ047654 for ; Mon, 10 Sep 2012 11:50:07 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q8ABo6oX047638; Mon, 10 Sep 2012 11:50:06 GMT (envelope-from gnats) Date: Mon, 10 Sep 2012 11:50:06 GMT Message-Id: <201209101150.q8ABo6oX047638@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Andrey Simonenko Cc: Subject: Re: kern/147881: [zfs] [patch] ZFS "sharenfs" doesn't allow different "exports" options for different hosts X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Andrey Simonenko List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Sep 2012 11:50:08 -0000 The following reply was made to PR kern/147881; it has been noted by GNATS. From: Andrey Simonenko To: bug-followup@FreeBSD.org Cc: Richard Conto , Martin Matuska Subject: Re: kern/147881: [zfs] [patch] ZFS "sharenfs" doesn't allow different "exports" options for different hosts Date: Mon, 10 Sep 2012 14:40:08 +0300 On Fri, Sep 07, 2012 at 06:10:07PM +0000, Richard Conto wrote: > > > The purpose of the sharenfs property is not to completely replace > > /etc/exports. > > What we have is just a tricky workaround that populates = > /etc/zfs/exports. > > T > > he commands to NFS-share filesystems on illumos are different and our > > options are mostly incompatible with illumos. > > So you e.g. sharenfs=3D"-maproot=3Droot" is invalid if imported on a > > Openindiana system. > >=20 > > Have you considered using just /etc/exports for more-complex = > configurations? > > Yes. It doesn't scale - and scaling is the whole purpose of NFS. It = > also makes it difficult for distributed management of access rights to = > end-user NFS exports. > > I wouldn't mind using a FreeBSD specific attribute to populate = > /etc/zfs/exports with FreeBSD specific values such as is used for = > swapping to ZFS. Merging the two different attributes would either be = > tricky, or the use of the FreeBSD one should cause "sharenfs" to be = > ignored entirely. This is a limitation of exports(5) format, that does not allow to specify all settings for one file system in one line. When I had similar task I allowed to redefine options in one line, so address specifications can inherit already specified settings and can get changed or new settings. For example: /fs -ro -mapall user1 host1 -mapall user2 host2 host3 -rw host4 Here, /fs is exported read-only for host{1,2,3} and read-write for host4, all users are mapped to user1 for host1 and all users are mapped to user2 for host{2,3,4}. More information about format of NFS exports settings that I propose is available here: http://nfse.sourceforge.net/nfs.exports.5.html From owner-freebsd-fs@FreeBSD.ORG Tue Sep 11 07:04:52 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 70E62106564A for ; Tue, 11 Sep 2012 07:04:52 +0000 (UTC) (envelope-from jurgen.weber@theiconic.com.au) Received: from exprod6og117.obsmtp.com (exprod6og117.obsmtp.com [64.18.1.39]) by mx1.freebsd.org (Postfix) with SMTP id 037ED8FC08 for ; Tue, 11 Sep 2012 07:04:51 +0000 (UTC) Received: from mail-iy0-f182.google.com ([209.85.210.182]) (using TLSv1) by exprod6ob117.postini.com ([64.18.5.12]) with SMTP ID DSNKUE7ikxjK7OfAAKGQMQNq60qkD8RGioV/@postini.com; Tue, 11 Sep 2012 00:04:52 PDT Received: by iayy25 with SMTP id y25so189019iay.13 for ; Tue, 11 Sep 2012 00:04:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:x-gm-message-state; bh=woRsgkHTAnixdXba68Iuf/fYtCVTqv+JyzkWxYOPeCE=; b=I08kjZthtR143DHy+k7Kaj7HkF81SREssuyPlcUe6JSC7lBKCsJzJVZa1lkgCmoPjH Z6AKBdyJRx8b3t1jpJVVabZeFownNpcIs0fp/ay7derbdVtYJcK/p7+d+hilhz4r7Cm4 nKYZvx1GRedfk1o1PKnoYNSMwQSQ7H6AWCm7aPxmuDjBwo1UaQFgAnJ4j8mzxQzJ16uv t75DcGY0YG07KmfDsy7RNHtKLusDtYzgn9F/a6chDUfQJN+vrGc4WiJE1BoudMDBb2/I PyhXARjl+B8mnAOiYuGiRuekW0AwdONJngWv74GaalzYm4Q2E3exMj+6KuNI7xlHCBRE wR6A== Received: by 10.50.173.69 with SMTP id bi5mr13604267igc.37.1347314302779; Mon, 10 Sep 2012 14:58:22 -0700 (PDT) Received: from [172.20.24.157] ([202.126.107.170]) by mx.google.com with ESMTPS id ua5sm328833igb.10.2012.09.10.14.58.21 (version=SSLv3 cipher=OTHER); Mon, 10 Sep 2012 14:58:22 -0700 (PDT) Message-ID: <504E627C.70408@theiconic.com.au> Date: Tue, 11 Sep 2012 07:58:20 +1000 From: Jurgen Weber User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120827 Thunderbird/15.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org X-Gm-Message-State: ALoCoQnT2QhLstY2+AtwvGAiVRV39gd+e4BmoBLauwuM0oNATpGVCyoHrCpagZEMoUrGP2+ymO1R Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: clustered file systems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 07:04:52 -0000 Guys I require a SAN solution for a mail server implementation I am designing. The design I am looking for is the following: https://www.communigate.com/cgatepro/ClusterDynamic.html#OSCluster This means I require some sort of SAN storage managed by a clustered FS, having very little SAN + ClusterFS experience I am research the options available to me. FreeBSD being my OS of choice but I found the following: http://blog.elitecoderz.net/cluster-filesystem-for-freebsd-gfs-ocfs2/2010/06/ This blogger mentions that their is no Clustered FS support at this point in time for FreeBSD which I find astonishing but I have been unable to find anything except for a GlusterFS port using fuse (not ideal). Any thoughts or suggestions appreciated. Thanks -- Jurgen Weber Systems Engineer IT Infrastructure Team Leader THE ICONIC | E jurgen.weber@theiconic.com.au | www.theiconic.com.au From owner-freebsd-fs@FreeBSD.ORG Tue Sep 11 08:20:19 2012 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6E4D9106564A for ; Tue, 11 Sep 2012 08:20:19 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 57D5B8FC0A for ; Tue, 11 Sep 2012 08:20:19 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q8B8KItG059176 for ; Tue, 11 Sep 2012 08:20:18 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q8B8KIIs059166; Tue, 11 Sep 2012 08:20:18 GMT (envelope-from gnats) Date: Tue, 11 Sep 2012 08:20:18 GMT Message-Id: <201209110820.q8B8KIIs059166@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Harald Schmalzbauer Cc: Subject: Re: kern/167105: [nfs] mount_nfs can not handle source exports wiht more then 63 chars X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Harald Schmalzbauer List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 08:20:19 -0000 The following reply was made to PR kern/167105; it has been noted by GNATS. From: Harald Schmalzbauer To: bug-followup@FreeBSD.org, msaad@about.com Cc: Subject: Re: kern/167105: [nfs] mount_nfs can not handle source exports wiht more then 63 chars Date: Tue, 11 Sep 2012 10:11:40 +0200 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigB2627C577733DCDF42FE5E42 Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: quoted-printable I've been hitten by the same constraint also for destionation node. Any news about the sense of this limit? Or if it's planned to relax it? Thanks, -Harry --------------enigB2627C577733DCDF42FE5E42 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAlBO8jwACgkQLDqVQ9VXb8ix7QCeJM1dOcsWS2uPqEgFV6hVTPh2 WiQAoM7bPCxin+RNABocVPqCi/STGlNG =G+sn -----END PGP SIGNATURE----- --------------enigB2627C577733DCDF42FE5E42-- From owner-freebsd-fs@FreeBSD.ORG Tue Sep 11 11:08:20 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 96838106566C for ; Tue, 11 Sep 2012 11:08:20 +0000 (UTC) (envelope-from Newsletter@goelli.de) Received: from mo6-p05-ob.rzone.de (mo6-p05-ob.rzone.de [IPv6:2a01:238:20a:202:5305::1]) by mx1.freebsd.org (Postfix) with ESMTP id EEE7E8FC08 for ; Tue, 11 Sep 2012 11:08:19 +0000 (UTC) X-RZG-CLASS-ID: mo05 X-RZG-AUTH: :ImkTZkytb+s5KUDumTG4i0mGDH1K4fweaf9O+/5rQT5ns8rb41Pk1sfUhKBRrQ== Received: from goelliNotebook (p4FEE52FB.dip.t-dialin.net [79.238.82.251]) by smtp.strato.de (jorabe mo11) (RZmta 30.14 DYNA|AUTH) with ESMTPA id 006b53o8BALJk8 for ; Tue, 11 Sep 2012 13:08:18 +0200 (CEST) From: =?iso-8859-1?Q?Thomas_G=F6llner_=28Newsletter=29?= To: Date: Tue, 11 Sep 2012 13:07:59 +0200 Message-ID: <001a01cd900d$bcfcc870$36f65950$@goelli.de> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 14.0 Thread-Index: Ac2QDbI7xL4yTxh5SqyFiLJJuz7H3Q== Content-Language: de Subject: ZFS: Corrupted pool metadata after adding vdev to a pool - no opportunity to rescue data from healthy vdevs? Remove a vdev? Rewrite metadata? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 11:08:20 -0000 Hi all, I recently crashed my pool with adding a new vdev to my pool. I=92m running NAS4Free 9.0.0.1 - Sandstorm (Revision 188). My Pool "GoelliZFS1" has one vdv - a raidz out of 3 discs =E1 3TB. As I needed = more space I put 3 discs =E1 1.5TB in the mashine and created a new raidz = vdev. Now something must have happened when I added the new vdev to the existing = pool. I think somehow the disclables got mixed up or something. Because after adding the vdev my pool had a capacity of 16TB o_O Until that point I = did everything via webGUI. I thought a restart could help, but after that my pool was gone. Now I did some reading and tried via CLI over SSH. I don't want to put = the whole log here, because it might be to long. I'll give shortup and if = you want to know more, just ask ;-) With "zpool import" I can see my pool. I checked the smart logs to = verify the disc names. Options -F and -X didn't help. With option -V the pool = was imported, but still faulty. goelli-nas4free:~# zpool import -faV goelli-nas4free:~# zpool status pool: GoelliZFS1 state: FAULTED status: The pool metadata is corrupted and the pool cannot be opened. action: Destroy and re-create the pool from a backup source. see: http://www.sun.com/msg/ZFS-8000-72 scan: none requested config: NAME STATE READ WRITE CKSUM GoelliZFS1 FAULTED 1 0 0 missing-0 ONLINE 0 0 0 raidz1-1 ONLINE 0 0 0 ada3 ONLINE 0 0 0 ada4 ONLINE 0 0 0 ada4 ONLINE 0 0 0 I used "zdb -l" for all discs. There are all 4 Labels on each disc. "zdb" also gave me some feedback (too long to post). I'm sure now, that my data is on the discs. After adding the new vdev I = had nothing changed. So there must be a chance to tell the zfs to dissmiss the wrong entry in = the metadata - or to edit the metadata myself... When I think of the following case, you would agree that there has to be = an opportunity to detach vdevs... If you have a pool of 4 vdevs, which is full of data, you are supposed = to add more space to the pool by adding a new vdev, right? If now, for some reason, after a short time and not much new data the new attached vdev completely fails - what do you do? ZFS is allways consistend on-disk. = ZFS has copy-on-write, so no data is changed until it's touched. In this = case you have a pool with 4 healthy vdevs with all you data and one faulty = vdev with almost no data. And you get the message to discard all your data, destroy the pool and roll-back from backup?! Somehow rediculous, right? I hope someone can tell me what I can try to do. I will appreachiate any kind of help... Greetings, Thomas From owner-freebsd-fs@FreeBSD.ORG Tue Sep 11 11:09:16 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7355C1065670 for ; Tue, 11 Sep 2012 11:09:16 +0000 (UTC) (envelope-from escholtz@argonsoft.de) Received: from coyote.quickmin.net (coyote.quickmin.net [217.14.112.24]) by mx1.freebsd.org (Postfix) with ESMTP id D3E698FC22 for ; Tue, 11 Sep 2012 11:09:15 +0000 (UTC) Received: (qmail 53646 invoked from network); 11 Sep 2012 13:02:33 +0200 Received: by simscan 1.4.0 ppid: 53643, pid: 53644, t: 0.0171s scanners: clamav: 0.97.5/m:54/d:15341 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=argonsoft.de; b=Xx8FP5PQshXROgpP+q1ugbXVRYp6e//OljRbSPGZHJX8hWSjHOi6ilTxGcKs9Q6sGmB7C3SCV3//huIFFxmH/FyBHryHI7XA9QBU+dLmkqj1PCC4rOebbbLQv5FWyaUbnLqo5IRHnvgsn3Y21coJs3KwOrrjIQgshbM/EDhTJm0= ; Received: from backup.quickmin.net (HELO hbv83020887.bskh.bauerverlag.de) (00000150@217.14.112.17) by coyote.quickmin.net with SMTP; 11 Sep 2012 13:02:33 +0200 Message-ID: <504F1A46.60509@argonsoft.de> Date: Tue, 11 Sep 2012 13:02:30 +0200 From: Erik Scholtz User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20120907 Thunderbird/15.0.1 MIME-Version: 1.0 To: Jurgen Weber References: <504E627C.70408@theiconic.com.au> In-Reply-To: <504E627C.70408@theiconic.com.au> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: clustered file systems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 11:09:16 -0000 Jurgen, I'm facing the same problem (that's why I wrote this article on my blog), and the only solution was to migrate from FreeBSD to CentOS (my choice - others are also possible). Up to now, there is no Cluster FS for FreeBSD and imho there isn't even a project with this target active. That's why I'm thinking, that the best OS in this world will die in future. Clustering features are key features these days. If anyone has another idea or information, I'm also very interested in this! Greetings, Erik -- My blog: http://blog.elitecoderz.net Am 10.09.12 23:58, schrieb Jurgen Weber: > Guys > > I require a SAN solution for a mail server implementation I am > designing. The design I am looking for is the following: > > https://www.communigate.com/cgatepro/ClusterDynamic.html#OSCluster > > This means I require some sort of SAN storage managed by a clustered FS, > having very little SAN + ClusterFS experience I am research the options > available to me. FreeBSD being my OS of choice but I found the following: > > http://blog.elitecoderz.net/cluster-filesystem-for-freebsd-gfs-ocfs2/2010/06/ > > > This blogger mentions that their is no Clustered FS support at this > point in time for FreeBSD which I find astonishing but I have been > unable to find anything except for a GlusterFS port using fuse (not ideal). > > Any thoughts or suggestions appreciated. > > Thanks > From owner-freebsd-fs@FreeBSD.ORG Tue Sep 11 12:01:55 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C05D0106566B for ; Tue, 11 Sep 2012 12:01:55 +0000 (UTC) (envelope-from c.kworr@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4D6D68FC12 for ; Tue, 11 Sep 2012 12:01:54 +0000 (UTC) Received: by eaak11 with SMTP id k11so243102eaa.13 for ; Tue, 11 Sep 2012 05:01:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=ANtpG24rEVAlDY4kHkgmgh1F1bSAAD1aCa0gk9dkkM8=; b=iAAvWcZQDZgAc9Ho3+vh18sc+levy8u2t5tLKwJtC2WVTYf9XUmT2y1/Iyg9HFKHhv JtQI03WrBSItIg0Bh/n+JegJ+SyZHriyriKIs9yfhyMt8h9KzqmulYpGwxHFY6sdFJ1y J7G0ieQQqxz46bl1viPgXr9ndFrawoPqjrxFOZ0Tn83Qwu4f9IypzHpaiu2PLPWBs7Cu 2+xpqs8FjQy5MjnWWhRyWH1Rpl+XXTAy6oehXrogMgRiNhK0/QzrQ5QE+V//AoFZ0+an Y6zjXt4pgfWeByanxoQ/SQYhBzqaJIFJlqcj89x0NxHQJcQ2BBdSYSbd2uH1wJ5Y1WgX c3mw== Received: by 10.14.209.5 with SMTP id r5mr23389277eeo.28.1347364913930; Tue, 11 Sep 2012 05:01:53 -0700 (PDT) Received: from green.local (189-29-132-95.pool.ukrtel.net. [95.132.29.189]) by mx.google.com with ESMTPS id k41sm46490010eep.13.2012.09.11.05.01.51 (version=SSLv3 cipher=OTHER); Tue, 11 Sep 2012 05:01:52 -0700 (PDT) Message-ID: <504F282D.8030808@gmail.com> Date: Tue, 11 Sep 2012 15:01:49 +0300 From: Volodymyr Kostyrko User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120831 Thunderbird/15.0 MIME-Version: 1.0 To: =?UTF-8?B?IlRob21hcyBHw7ZsbG5lciAoTmV3c2xldHRlciki?= References: <001a01cd900d$bcfcc870$36f65950$@goelli.de> In-Reply-To: <001a01cd900d$bcfcc870$36f65950$@goelli.de> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-fs@freebsd.org Subject: Re: ZFS: Corrupted pool metadata after adding vdev to a pool - no opportunity to rescue data from healthy vdevs? Remove a vdev? Rewrite metadata? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 12:01:55 -0000 11.09.2012 14:07, Thomas Göllner (Newsletter) wrote: > I hope someone can tell me what I can try to do. I will appreachiate any > kind of help... Looks pretty bad, but you can try readonly mode with transaction rollback and recheck: zpool import -f -R /GoelliZFS1 -N -o readonly=on -F -X GoelliZFS1 Now by options: -f: force import; -R: change root directory for all mounts; -N: do not mount filesystems; -o: pass extra options to zfs layer; -F: try to discard last few transactions; -X: check thoroughly each transaction skipped (undocumented). You can skip -X for the first time, with -X it can take tens of hours. If this doesn't help I welcome you to our club of 'I borged my ZFS'. I lost my pool after fiddling with my hardware so last few transactions got damaged and I constantly see only yell on 'bad objset' but I can't mount any other objset. -- Sphinx of black quartz judge my vow. From owner-freebsd-fs@FreeBSD.ORG Tue Sep 11 15:21:04 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C2AD3106564A for ; Tue, 11 Sep 2012 15:21:04 +0000 (UTC) (envelope-from gdt@ir.bbn.com) Received: from fnord.ir.bbn.com (fnord.ir.bbn.com [IPv6:2001:4978:1fb:6400::d2]) by mx1.freebsd.org (Postfix) with ESMTP id 953FB8FC15 for ; Tue, 11 Sep 2012 15:21:04 +0000 (UTC) Received: by fnord.ir.bbn.com (Postfix, from userid 10853) id E4DEFBE89; Tue, 11 Sep 2012 11:21:03 -0400 (EDT) From: Greg Troxel To: Jurgen Weber References: <504E627C.70408@theiconic.com.au> OpenPGP: id=32611E25 X-Hashcash: 1:20:120911:jurgen.weber@theiconic.com.au::UN/WETug1VqGy73U:0000000000000000000000000000000006HD X-Hashcash: 1:20:120911:freebsd-fs@freebsd.org::5Aq+2WVEy7oyGuqm:000000000000000000000000000000000000000AK2S Date: Tue, 11 Sep 2012 11:21:03 -0400 In-Reply-To: <504E627C.70408@theiconic.com.au> (Jurgen Weber's message of "Tue, 11 Sep 2012 07:58:20 +1000") Message-ID: User-Agent: Gnus/5.130006 (Ma Gnus v0.6) Emacs/23.4 (berkeley-unix) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Cc: freebsd-fs@freebsd.org Subject: Re: clustered file systems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 15:21:04 -0000 --=-=-= Content-Type: text/plain Jurgen Weber writes: > Guys > > I require a SAN solution for a mail server implementation I am > designing. The design I am looking for is the following: > > https://www.communigate.com/cgatepro/ClusterDynamic.html#OSCluster > > This means I require some sort of SAN storage managed by a clustered > FS, having very little SAN + ClusterFS experience I am research the > options available to me. FreeBSD being my OS of choice but I found the > following: > > http://blog.elitecoderz.net/cluster-filesystem-for-freebsd-gfs-ocfs2/2010/06/ > > This blogger mentions that their is no Clustered FS support at this > point in time for FreeBSD which I find astonishing but I have been > unable to find anything except for a GlusterFS port using fuse (not > ideal). On NetBSD, I have the impression from list reading that glusterfs works well with perfuse/puffs (filling most of GbE for reads). I realize NetBSD's puffs/fuse implementation is different, but people may want to look at it for porting to FreeBSD. --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (NetBSD) iEYEARECAAYFAlBPVt8ACgkQ+vesoDJhHiWBqwCgkbUIxkt/wBqfalZCG3JbycxL xJkAn3OJBw5H7KIL4MRP62ZlMnoQjUrd =v0P6 -----END PGP SIGNATURE----- --=-=-=-- From owner-freebsd-fs@FreeBSD.ORG Tue Sep 11 16:10:58 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 673F71065673 for ; Tue, 11 Sep 2012 16:10:58 +0000 (UTC) (envelope-from nicolas@i.0x5.de) Received: from n.0x5.de (n.0x5.de [217.197.85.144]) by mx1.freebsd.org (Postfix) with ESMTP id 086A38FC22 for ; Tue, 11 Sep 2012 16:10:56 +0000 (UTC) Received: by pc5.i.0x5.de (Postfix, from userid 1003) id 3XGWLx66Npz7ySD; Tue, 11 Sep 2012 18:10:49 +0200 (CEST) Date: Tue, 11 Sep 2012 18:10:49 +0200 From: Nicolas Rachinsky To: freebsd-fs@freebsd.org Message-ID: <20120911161049.GA21635@mid.pc5.i.0x5.de> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="J/dobhs11T7y2rNN" Content-Disposition: inline X-Powered-by: FreeBSD X-Homepage: http://www.rachinsky.de X-PGP-Keyid: 887BAE72 X-PGP-Fingerprint: 039E 9433 115F BC5F F88D 4524 5092 45C4 887B AE72 X-PGP-Keys: http://www.rachinsky.de/nicolas/gpg/nicolas_rachinsky.asc User-Agent: Mutt/1.5.21 (2010-09-15) Subject: bug? with zfs, symlinks and extended attributes X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 16:10:58 -0000 --J/dobhs11T7y2rNN Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hallo, I can reliable "break" zfs on FreeBSD 8.3-RELEASE-p4 (on two different machines). I just do the following: zpool create pool1 ada0p15 zfs create pool1/TST cd /pool1/TST sh attached script cd - zfs umount pool1/TST zfs mount pool1/TST Afterwards some of the symlinks in /pool1/TST have extended attributes I've never set. Others cannot have any extended attributes. For example: # lsextattr -h user /pool1/TST/etc/rc0.d/K04rsyslog lsextattr: /pool1/TST/etc/rc0.d/K04rsyslog: failed: Invalid argument setextattr fails as well. Not all symlinks show this problem. The set showing the problem seems to be not always the same. With the following changes to the attached script, I cannot reproduce the problem: leaving out all touch-lines. set all extended attributes to a string of length 1. let all symlinks point to foo. The problem does occur when I run the script as normal user. Nicolas -- http://www.rachinsky.de/nicolas --J/dobhs11T7y2rNN Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="break.sh" mkdir etc mkdir etc/Muttrc.d mkdir etc/X11 mkdir etc/X11/xkb mkdir etc/acpi mkdir etc/acpi/events mkdir etc/alternatives mkdir etc/apt mkdir etc/apt/apt.conf.d mkdir etc/apt/preferences.d mkdir etc/apt/sources.list.d mkdir etc/apt/trusted.gpg.d mkdir etc/bash_completion.d mkdir etc/ca-certificates mkdir etc/ca-certificates/update.d mkdir etc/calendar mkdir etc/console-setup mkdir etc/cron.d mkdir etc/cron.daily mkdir etc/cron.hourly mkdir etc/cron.monthly mkdir etc/cron.weekly mkdir etc/default mkdir etc/dhcp mkdir etc/dhcp/dhclient-enter-hooks.d mkdir etc/dhcp/dhclient-exit-hooks.d mkdir etc/discover.conf.d mkdir etc/dpkg mkdir etc/dpkg/dpkg.cfg.d mkdir etc/dpkg/origins mkdir etc/exim4 mkdir etc/exim4/conf.d mkdir etc/exim4/conf.d/acl mkdir etc/exim4/conf.d/auth mkdir etc/exim4/conf.d/main mkdir etc/exim4/conf.d/retry mkdir etc/exim4/conf.d/rewrite mkdir etc/exim4/conf.d/router mkdir etc/exim4/conf.d/transport mkdir etc/groff mkdir etc/grub.d mkdir etc/ha.d mkdir etc/ha.d/rc.d mkdir etc/ha.d/resource.d mkdir etc/init.d mkdir etc/insserv mkdir etc/insserv/overrides mkdir etc/initramfs-tools mkdir etc/initramfs-tools/conf.d mkdir etc/initramfs-tools/hooks mkdir etc/initramfs-tools/scripts mkdir etc/initramfs-tools/scripts/init-bottom mkdir etc/initramfs-tools/scripts/init-premount mkdir etc/initramfs-tools/scripts/init-top mkdir etc/initramfs-tools/scripts/local-bottom mkdir etc/initramfs-tools/scripts/local-premount mkdir etc/initramfs-tools/scripts/local-top mkdir etc/initramfs-tools/scripts/nfs-bottom mkdir etc/initramfs-tools/scripts/nfs-premount mkdir etc/initramfs-tools/scripts/nfs-top mkdir etc/insserv.conf.d mkdir etc/iproute2 mkdir etc/kbd mkdir etc/kernel mkdir etc/kernel/postinst.d mkdir etc/kernel/postrm.d mkdir etc/ld.so.conf.d mkdir etc/ldap mkdir etc/logcheck mkdir etc/logcheck/ignore.d.server mkdir etc/logrotate.d mkdir etc/lsb-base mkdir etc/lvm mkdir etc/lvm/archive mkdir etc/lvm/backup mkdir etc/lvm/cache mkdir etc/modprobe.d mkdir etc/mysql-proxy mkdir etc/mysql mkdir etc/mysql/conf.d mkdir etc/network mkdir etc/network/if-down.d mkdir etc/network/if-post-down.d mkdir etc/network/if-pre-up.d mkdir etc/network/if-up.d mkdir etc/network/run mkdir etc/openhpi mkdir etc/opt mkdir etc/pam.d mkdir etc/perl mkdir etc/perl/CPAN mkdir etc/perl/Net mkdir etc/ppp mkdir etc/ppp/ip-up.d mkdir etc/profile.d mkdir etc/python mkdir etc/python2.6 mkdir etc/rc0.d mkdir etc/rc1.d mkdir etc/rc2.d mkdir etc/rc3.d mkdir etc/rc4.d mkdir etc/rc5.d mkdir etc/rc6.d mkdir etc/rcS.d mkdir etc/rsyslog.d mkdir etc/security mkdir etc/security/limits.d mkdir etc/security/namespace.d mkdir etc/sensors.d mkdir etc/sgml mkdir etc/skel mkdir etc/snmp mkdir etc/ssh mkdir etc/ssl mkdir etc/ssl/certs mkdir etc/ssl/private mkdir etc/sysctl.d mkdir etc/terminfo mkdir etc/texmf mkdir etc/texmf/fmt.d mkdir etc/udev mkdir etc/udev/.dev mkdir etc/udev/rules.d mkdir etc/ufw mkdir etc/ufw/applications.d mkdir etc/vim mkdir etc/w3m mkdir etc/xml touch etc/Muttrc.d/charset.rc touch etc/Muttrc.d/colors.rc touch etc/Muttrc.d/gpg.rc touch etc/Muttrc.d/smime.rc touch etc/Muttrc.d/compressed-folders.rc touch etc/acpi/events/powerbtn-acpi-support touch etc/acpi/powerbtn-acpi-support.sh touch etc/alternatives/README touch etc/apt/apt.conf.d/00CDMountPoint touch etc/apt/apt.conf.d/00trustcdrom touch etc/apt/apt.conf.d/01autoremove touch etc/apt/apt.conf.d/20listchanges touch etc/apt/apt.conf.d/70debconf touch etc/apt/listchanges.conf touch etc/apt/apt-file.conf touch etc/apt/sources.list touch etc/apt/secring.gpg touch etc/apt/sources.list~ touch etc/apt/trustdb.gpg touch etc/apt/trusted.gpg touch etc/apt/trusted.gpg~ touch etc/bash_completion.d/apache2ctl touch etc/bash_completion.d/abook touch etc/bash_completion.d/ant touch etc/bash_completion.d/apt-build touch etc/bash_completion.d/apt touch etc/bash_completion.d/bash-builtins touch etc/bash_completion.d/apt-file touch etc/bash_completion.d/aptitude touch etc/bash_completion.d/aspell touch etc/bash_completion.d/autorpm touch etc/bash_completion.d/initramfs-tools touch etc/bash_completion.d/bind-utils touch etc/bash_completion.d/bitkeeper touch etc/bash_completion.d/bittorrent touch etc/bash_completion.d/bluez touch etc/bash_completion.d/brctl touch etc/bash_completion.d/bzip2 touch etc/bash_completion.d/cardctl touch etc/bash_completion.d/cfengine touch etc/bash_completion.d/chkconfig touch etc/bash_completion.d/chsh touch etc/bash_completion.d/cksfv touch etc/bash_completion.d/clisp touch etc/bash_completion.d/configure touch etc/bash_completion.d/coreutils touch etc/bash_completion.d/cowsay touch etc/bash_completion.d/info touch etc/bash_completion.d/cpan2dist touch etc/bash_completion.d/cpio touch etc/bash_completion.d/cryptsetup touch etc/bash_completion.d/cups touch etc/bash_completion.d/cvs touch etc/bash_completion.d/cvsps touch etc/bash_completion.d/dd touch etc/bash_completion.d/debconf touch etc/bash_completion.d/dhclient touch etc/bash_completion.d/dict touch etc/bash_completion.d/dpkg touch etc/bash_completion.d/dselect touch etc/bash_completion.d/dsniff touch etc/bash_completion.d/findutils touch etc/bash_completion.d/freeciv touch etc/bash_completion.d/fuse touch etc/bash_completion.d/gcc touch etc/bash_completion.d/gcl touch etc/bash_completion.d/gdb touch etc/bash_completion.d/genisoimage touch etc/bash_completion.d/getent touch etc/bash_completion.d/gkrellm touch etc/bash_completion.d/gnatmake touch etc/bash_completion.d/gpg touch etc/bash_completion.d/gpg2 touch etc/bash_completion.d/gzip touch etc/bash_completion.d/heimdal touch etc/bash_completion.d/hping2 touch etc/bash_completion.d/iconv touch etc/bash_completion.d/ifupdown touch etc/bash_completion.d/imagemagick touch etc/bash_completion.d/ipmitool touch etc/bash_completion.d/insserv touch etc/bash_completion.d/iptables touch etc/bash_completion.d/ipsec touch etc/bash_completion.d/perl touch etc/bash_completion.d/p4 touch etc/bash_completion.d/ipv6calc touch etc/bash_completion.d/isql touch etc/bash_completion.d/jar touch etc/bash_completion.d/java touch etc/bash_completion.d/k3b touch etc/bash_completion.d/kldload touch etc/bash_completion.d/larch touch etc/bash_completion.d/ldapvi touch etc/bash_completion.d/lftp touch etc/bash_completion.d/lilo touch etc/bash_completion.d/links touch etc/bash_completion.d/lintian touch etc/bash_completion.d/lisp touch etc/bash_completion.d/lvm touch etc/bash_completion.d/lzma touch etc/bash_completion.d/lzop touch etc/bash_completion.d/mailman touch etc/bash_completion.d/make touch etc/bash_completion.d/man touch etc/bash_completion.d/mc touch etc/bash_completion.d/mcrypt touch etc/bash_completion.d/mdadm touch etc/bash_completion.d/medusa touch etc/bash_completion.d/minicom touch etc/bash_completion.d/mkinitrd touch etc/bash_completion.d/mount touch etc/bash_completion.d/module-init-tools touch etc/bash_completion.d/monodevelop touch etc/bash_completion.d/mplayer touch etc/bash_completion.d/msynctool touch etc/bash_completion.d/munin-node touch etc/bash_completion.d/mutt touch etc/bash_completion.d/mysqladmin touch etc/bash_completion.d/ncftp touch etc/bash_completion.d/net-tools touch etc/bash_completion.d/nmap touch etc/bash_completion.d/ntpdate touch etc/bash_completion.d/openldap touch etc/bash_completion.d/openssl touch etc/bash_completion.d/pkg-config touch etc/bash_completion.d/pine touch etc/bash_completion.d/rdesktop touch etc/bash_completion.d/rcs touch etc/bash_completion.d/pkg_install touch etc/bash_completion.d/pkgtools touch etc/bash_completion.d/pm-utils touch etc/bash_completion.d/portupgrade touch etc/bash_completion.d/postfix touch etc/bash_completion.d/postgresql touch etc/bash_completion.d/povray touch etc/bash_completion.d/procps touch etc/bash_completion.d/python touch etc/bash_completion.d/qdbus touch etc/bash_completion.d/qemu touch etc/bash_completion.d/quota-tools touch etc/bash_completion.d/reportbug touch etc/bash_completion.d/resolvconf touch etc/bash_completion.d/rfkill touch etc/bash_completion.d/ri touch etc/bash_completion.d/rpcdebug touch etc/bash_completion.d/rpm touch etc/bash_completion.d/rpmcheck touch etc/bash_completion.d/rrdtool touch etc/bash_completion.d/rsync touch etc/bash_completion.d/rtcwake touch etc/bash_completion.d/samba touch etc/bash_completion.d/sbcl touch etc/bash_completion.d/screen touch etc/bash_completion.d/service touch etc/bash_completion.d/shadow touch etc/bash_completion.d/sitecopy touch etc/bash_completion.d/smartctl touch etc/bash_completion.d/snownews touch etc/bash_completion.d/ssh touch etc/bash_completion.d/sshfs touch etc/bash_completion.d/strace touch etc/bash_completion.d/svk touch etc/bash_completion.d/sysctl touch etc/bash_completion.d/sysv-rc touch etc/bash_completion.d/tar touch etc/bash_completion.d/tcpdump touch etc/bash_completion.d/unace touch etc/bash_completion.d/unrar touch etc/bash_completion.d/wol touch etc/bash_completion.d/xsltproc touch etc/bash_completion.d/update-alternatives touch etc/bash_completion.d/util-linux touch etc/bash_completion.d/vncviewer touch etc/bash_completion.d/vpnc touch etc/bash_completion.d/wireless-tools touch etc/bash_completion.d/wodim touch etc/bash_completion.d/wtf touch etc/bash_completion.d/wvdial touch etc/bash_completion.d/xhost touch etc/bash_completion.d/xm touch etc/bash_completion.d/xmllint touch etc/bash_completion.d/xmlwf touch etc/bash_completion.d/xmms touch etc/bash_completion.d/xrandr touch etc/bash_completion.d/xz touch etc/bash_completion.d/yp-tools touch etc/bash_completion.d/yum-arch touch etc/calendar/default touch etc/console-setup/compose.GEORGIAN-ACADEMY.inc touch etc/console-setup/Lat15-Fixed16.psf.gz touch etc/console-setup/cached.kmap.gz touch etc/console-setup/compose.ARMSCII-8.inc touch etc/console-setup/compose.CP1251.inc touch etc/console-setup/compose.CP1255.inc touch etc/console-setup/compose.CP1256.inc touch etc/console-setup/compose.GEORGIAN-PS.inc touch etc/console-setup/compose.IBM1133.inc touch etc/console-setup/compose.ISIRI-3342.inc touch etc/console-setup/compose.ISO-8859-1.inc touch etc/console-setup/compose.ISO-8859-10.inc touch etc/console-setup/compose.ISO-8859-11.inc touch etc/console-setup/compose.ISO-8859-13.inc touch etc/console-setup/compose.ISO-8859-14.inc touch etc/console-setup/compose.ISO-8859-15.inc touch etc/console-setup/compose.ISO-8859-16.inc touch etc/console-setup/compose.ISO-8859-2.inc touch etc/console-setup/compose.ISO-8859-3.inc touch etc/console-setup/compose.ISO-8859-4.inc touch etc/console-setup/compose.ISO-8859-5.inc touch etc/console-setup/compose.ISO-8859-6.inc touch etc/console-setup/compose.ISO-8859-7.inc touch etc/console-setup/compose.ISO-8859-8.inc touch etc/console-setup/compose.ISO-8859-9.inc touch etc/console-setup/compose.KOI8-R.inc touch etc/console-setup/compose.KOI8-U.inc touch etc/console-setup/compose.TIS-620.inc touch etc/console-setup/compose.VISCII.inc touch etc/console-setup/remap.inc touch etc/cron.d/.placeholder touch etc/cron.daily/aptitude touch etc/cron.daily/apt touch etc/cron.daily/.placeholder touch etc/cron.daily/bsdmainutils touch etc/cron.daily/dpkg touch etc/cron.daily/exim4-base touch etc/cron.daily/logrotate touch etc/cron.daily/man-db touch etc/cron.daily/mlocate touch etc/cron.daily/passwd touch etc/cron.daily/standard touch etc/cron.hourly/.placeholder touch etc/cron.monthly/.placeholder touch etc/cron.weekly/.placeholder touch etc/cron.weekly/man-db touch etc/default/bootlogd touch etc/default/acpid touch etc/default/rsync touch etc/default/rcS touch etc/default/console-setup touch etc/default/cron touch etc/default/devpts touch etc/default/exim4 touch etc/default/grub touch etc/default/halt touch etc/default/ifupdown touch etc/default/keyboard touch etc/default/locale touch etc/default/mysql-proxy touch etc/default/nfs-common touch etc/default/portmap touch etc/default/rsyslog touch etc/default/ssh touch etc/default/tmpfs touch etc/default/useradd touch etc/dhcp/dhclient-enter-hooks.d/debug touch etc/dhcp/dhclient-exit-hooks.d/debug touch etc/dhcp/dhclient-exit-hooks.d/rfc3442-classless-routes touch etc/dhcp/dhclient.conf touch etc/discover.conf.d/00discover touch etc/dpkg/origins/debian touch etc/dpkg/dpkg.cfg touch etc/exim4/conf.d/acl/30_exim4-config_check_mail touch etc/exim4/conf.d/acl/00_exim4-config_header touch etc/exim4/conf.d/acl/20_exim4-config_local_deny_exceptions touch etc/exim4/conf.d/acl/30_exim4-config_check_rcpt touch etc/exim4/conf.d/acl/40_exim4-config_check_data touch etc/exim4/conf.d/auth/30_exim4-config_examples touch etc/exim4/conf.d/auth/00_exim4-config_header touch etc/exim4/conf.d/main/01_exim4-config_listmacrosdefs touch etc/exim4/conf.d/main/02_exim4-config_options touch etc/exim4/conf.d/main/03_exim4-config_tlsoptions touch etc/exim4/conf.d/main/90_exim4-config_log_selector touch etc/exim4/conf.d/retry/00_exim4-config_header touch etc/exim4/conf.d/retry/30_exim4-config touch etc/exim4/conf.d/rewrite/31_exim4-config_rewriting touch etc/exim4/conf.d/rewrite/00_exim4-config_header touch etc/exim4/conf.d/router/100_exim4-config_domain_literal touch etc/exim4/conf.d/router/00_exim4-config_header touch etc/exim4/conf.d/router/150_exim4-config_hubbed_hosts touch etc/exim4/conf.d/router/200_exim4-config_primary touch etc/exim4/conf.d/router/300_exim4-config_real_local touch etc/exim4/conf.d/router/400_exim4-config_system_aliases touch etc/exim4/conf.d/router/500_exim4-config_hubuser touch etc/exim4/conf.d/router/600_exim4-config_userforward touch etc/exim4/conf.d/router/700_exim4-config_procmail touch etc/exim4/conf.d/router/800_exim4-config_maildrop touch etc/exim4/conf.d/router/850_exim4-config_lowuid touch etc/exim4/conf.d/router/900_exim4-config_local_user touch etc/exim4/conf.d/router/mmm_mail4root touch etc/exim4/conf.d/transport/30_exim4-config_address_file touch etc/exim4/conf.d/transport/00_exim4-config_header touch etc/exim4/conf.d/transport/10_exim4-config_transport-macros touch etc/exim4/conf.d/transport/30_exim4-config_address_pipe touch etc/exim4/conf.d/transport/30_exim4-config_address_reply touch etc/exim4/conf.d/transport/30_exim4-config_mail_spool touch etc/exim4/conf.d/transport/30_exim4-config_maildir_home touch etc/exim4/conf.d/transport/30_exim4-config_maildrop_pipe touch etc/exim4/conf.d/transport/30_exim4-config_procmail_pipe touch etc/exim4/conf.d/transport/30_exim4-config_remote_smtp touch etc/exim4/conf.d/transport/35_exim4-config_address_directory touch etc/exim4/conf.d/transport/30_exim4-config_remote_smtp_smarthost touch etc/exim4/update-exim4.conf.conf touch etc/exim4/exim4.conf.template touch etc/exim4/passwd.client touch etc/groff/man.local touch etc/groff/mdoc.local touch etc/grub.d/05_debian_theme touch etc/grub.d/00_header touch etc/grub.d/20_linux_xen touch etc/grub.d/10_linux touch etc/grub.d/30_os-prober touch etc/grub.d/40_custom touch etc/grub.d/41_custom touch etc/grub.d/README touch etc/ha.d/rc.d/ask_resources touch etc/ha.d/rc.d/hb_takeover touch etc/ha.d/rc.d/ip-request touch etc/ha.d/rc.d/ip-request-resp touch etc/ha.d/rc.d/status touch etc/ha.d/resource.d/IPaddr touch etc/ha.d/resource.d/ICP touch etc/ha.d/resource.d/AudibleAlarm touch etc/ha.d/resource.d/Delay touch etc/ha.d/resource.d/Filesystem touch etc/ha.d/resource.d/IPsrcaddr touch etc/ha.d/resource.d/IPaddr2 touch etc/ha.d/resource.d/hto-mapfuncs touch etc/ha.d/resource.d/IPv6addr touch etc/ha.d/resource.d/LVM touch etc/ha.d/resource.d/LinuxSCSI touch etc/ha.d/resource.d/MailTo touch etc/ha.d/resource.d/OCF touch etc/ha.d/resource.d/Raid1 touch etc/ha.d/resource.d/SendArp touch etc/ha.d/resource.d/ServeRAID touch etc/ha.d/resource.d/WAS touch etc/ha.d/resource.d/WinPopup touch etc/ha.d/resource.d/Xinetd touch etc/ha.d/resource.d/apache touch etc/ha.d/resource.d/db2 touch etc/ha.d/resource.d/mysql-proxy touch etc/ha.d/resource.d/ids touch etc/ha.d/resource.d/portblock touch etc/ha.d/README.config touch etc/ha.d/authkeys touch etc/ha.d/ha.cf touch etc/ha.d/harc touch etc/ha.d/haresources touch etc/ha.d/shellfuncs touch etc/init.d/bootlogd touch etc/init.d/atd touch etc/init.d/.depend.boot touch etc/init.d/.depend.start touch etc/init.d/.depend.stop touch etc/init.d/README touch etc/init.d/acpid touch etc/init.d/checkroot.sh touch etc/init.d/bootlogs touch etc/init.d/bootmisc.sh touch etc/init.d/checkfs.sh touch etc/init.d/kbd touch etc/init.d/console-setup touch etc/init.d/cron touch etc/init.d/exim4 touch etc/init.d/fancontrol touch etc/init.d/halt touch etc/init.d/heartbeat touch etc/init.d/hostname.sh touch etc/init.d/hwclock.sh touch etc/init.d/hwclockfirst.sh touch etc/init.d/ifupdown touch etc/init.d/ifupdown-clean touch etc/init.d/mountall-bootclean.sh touch etc/init.d/keyboard-setup touch etc/init.d/killprocs touch etc/init.d/lm-sensors touch etc/init.d/logd touch etc/init.d/lvm2 touch etc/init.d/module-init-tools touch etc/init.d/mountdevsubfs.sh touch etc/init.d/mountall.sh touch etc/init.d/mountnfs-bootclean.sh touch etc/init.d/mountkernfs.sh touch etc/init.d/mountoverflowtmp touch etc/init.d/mountnfs.sh touch etc/init.d/mysql-proxy touch etc/init.d/mtab.sh touch etc/init.d/rc.local touch etc/init.d/mysql-proxy.old touch etc/init.d/mysql.server touch etc/init.d/ndb_mgmd touch etc/init.d/ndbd touch etc/init.d/networking touch etc/init.d/nfs-common touch etc/init.d/openhpid touch etc/init.d/portmap touch etc/init.d/procps touch etc/init.d/rc touch etc/init.d/reboot touch etc/init.d/rcS touch etc/init.d/screen-cleanup touch etc/init.d/rmnologin touch etc/init.d/rsync touch etc/init.d/rsyslog touch etc/init.d/stop-bootlogd touch etc/init.d/sendsigs touch etc/init.d/single touch etc/init.d/skeleton touch etc/init.d/ssh touch etc/init.d/udev-mtab touch etc/init.d/udev touch etc/init.d/stop-bootlogd-single touch etc/init.d/umountnfs.sh touch etc/init.d/umountfs touch etc/init.d/umountroot touch etc/init.d/urandom touch etc/initramfs-tools/update-initramfs.conf touch etc/initramfs-tools/initramfs.conf touch etc/initramfs-tools/modules touch etc/iproute2/ematch_map touch etc/iproute2/rt_dsfield touch etc/iproute2/rt_protos touch etc/iproute2/rt_realms touch etc/iproute2/rt_scopes touch etc/iproute2/rt_tables touch etc/kbd/config touch etc/kbd/remap touch etc/kernel/postinst.d/initramfs-tools touch etc/kernel/postinst.d/zz-update-grub touch etc/kernel/postrm.d/initramfs-tools touch etc/kernel/postrm.d/zz-update-grub touch etc/ld.so.conf.d/libc.conf touch etc/ld.so.conf.d/x86_64-linux-gnu.conf touch etc/ldap/ldap.conf touch etc/logcheck/ignore.d.server/rsyslog touch etc/logrotate.d/aptitude touch etc/logrotate.d/apt touch etc/logrotate.d/exim4-base touch etc/logrotate.d/dpkg touch etc/logrotate.d/exim4-paniclog touch etc/logrotate.d/heartbeat touch etc/logrotate.d/rsyslog touch etc/lvm/archive/db1store_00000.vg touch etc/lvm/archive/db1store_00001.vg touch etc/lvm/archive/db1store_00002.vg touch etc/lvm/backup/db1store touch etc/lvm/cache/.cache touch etc/lvm/lvm.conf touch etc/modprobe.d/fbdev-blacklist.conf touch etc/modprobe.d/aliases.conf touch etc/modprobe.d/blacklist.conf touch etc/mysql-proxy/mysql-proxy.conf touch etc/mysql/my.cnf-disabled touch etc/network/if-post-down.d/ifenslave touch etc/network/if-pre-up.d/ifenslave touch etc/network/if-up.d/openssh-server touch etc/network/if-up.d/ifenslave touch etc/network/if-up.d/mountnfs touch etc/network/run/ifstate touch etc/network/interfaces touch etc/openhpi/openhpiclient.conf touch etc/openhpi/openhpi.conf touch etc/pam.d/chfn touch etc/pam.d/atd touch etc/pam.d/common-account touch etc/pam.d/chpasswd touch etc/pam.d/chsh touch etc/pam.d/common-password touch etc/pam.d/common-auth touch etc/pam.d/su touch etc/pam.d/common-session touch etc/pam.d/cron touch etc/pam.d/login touch etc/pam.d/other touch etc/pam.d/common-session-noninteractive touch etc/pam.d/newusers touch etc/pam.d/passwd touch etc/pam.d/sshd touch etc/perl/Net/libnet.cfg touch etc/ppp/ip-up.d/exim4 touch etc/profile.d/bash_completion.sh touch etc/python/debian_config touch etc/python2.6/sitecustomize.py touch etc/rc0.d/README touch etc/rc1.d/README touch etc/rc2.d/README touch etc/rc3.d/README touch etc/rc4.d/README touch etc/rc5.d/README touch etc/rc6.d/README touch etc/rcS.d/README touch etc/security/namespace.conf touch etc/security/access.conf touch etc/security/group.conf touch etc/security/limits.conf touch etc/security/namespace.init touch etc/security/opasswd touch etc/security/pam_env.conf touch etc/security/sepermit.conf touch etc/security/time.conf touch etc/sensors.d/.placeholder touch etc/sgml/catalog.old touch etc/sgml/catalog touch etc/sgml/xml-core.cat touch etc/skel/.bash_logout touch etc/skel/.bashrc touch etc/skel/.profile touch etc/snmp/snmp.conf touch etc/ssh/ssh_config touch etc/ssh/moduli touch etc/ssh/ssh_host_dsa_key.pub touch etc/ssh/ssh_host_dsa_key touch etc/ssh/ssh_host_rsa_key.pub touch etc/ssh/ssh_host_rsa_key touch etc/ssh/sshd_config touch etc/ssl/certs/ca-certificates.crt touch etc/ssl/openssl.cnf touch etc/sysctl.d/README.sysctl touch etc/terminfo/README touch etc/texmf/fmt.d/50cyrtexinfo.cnf touch etc/udev/rules.d/70-persistent-cd.rules touch etc/udev/rules.d/70-persistent-net.rules touch etc/udev/links.conf touch etc/udev/udev.conf touch etc/ufw/applications.d/openssh-server touch etc/vim/vimrc.tiny touch etc/vim/vimrc touch etc/w3m/config touch etc/w3m/mailcap touch etc/xml/catalog.old touch etc/xml/catalog touch etc/xml/xml-core.xml.old touch etc/xml/xml-core.xml touch etc/adduser.conf touch etc/.pwd.lock touch etc/Muttrc touch etc/bash.bashrc touch etc/aliases touch etc/at.deny touch etc/bindresvport.blacklist touch etc/bash_completion touch etc/debconf.conf touch etc/crontab touch etc/issue.net touch etc/ca-certificates.conf touch etc/discover-modprobe.conf touch etc/debian_version touch etc/deluser.conf touch etc/gssapi_mech.conf touch etc/email-addresses touch etc/environment touch etc/fstab touch etc/gai.conf touch etc/group touch etc/group- touch etc/gshadow touch etc/gshadow- touch etc/insserv.conf touch etc/host.conf touch etc/hostname touch etc/hosts touch etc/hosts.allow touch etc/hosts.deny touch etc/idmapd.conf touch etc/inittab touch etc/inputrc touch etc/rmt touch etc/issue touch etc/rsyslog.conf touch etc/kernel-img.conf touch etc/ld.so.cache touch etc/ld.so.conf touch etc/locale.alias touch etc/locale.gen touch etc/localtime touch etc/login.defs touch etc/logrotate.conf touch etc/magic touch etc/magic.mime touch etc/mail.rc touch etc/mailcap touch etc/mailcap.order touch etc/mailname touch etc/manpath.config touch etc/mime.types touch etc/mke2fs.conf touch etc/modules touch etc/motd.tail touch etc/mtab touch etc/my.cnf touch etc/nanorc touch etc/networks touch etc/nsswitch.conf touch etc/pam.conf touch etc/passwd touch etc/passwd- touch etc/profile touch etc/protocols touch etc/rc.local touch etc/reportbug.conf touch etc/resolv.conf touch etc/rpc touch etc/screenrc touch etc/securetty touch etc/sensors3.conf touch etc/services touch etc/shadow touch etc/shadow- touch etc/shells touch etc/sysctl.conf touch etc/timezone touch etc/ucf.conf touch etc/updatedb.conf touch etc/wgetrc ln -s xxxx etc/heartbeat ln -s xxxxxxxxxxxxx etc/motd ln -s xxxxxxxxxxxxxxxxxx etc/alternatives/Mail ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/Mail.1.gz ln -s xxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/aptitude ln -s xxxxxxxxxxxxx etc/alternatives/awk ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/awk.1.gz ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/builtins.7.gz ln -s xxxxxxxxx etc/alternatives/editor ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/editor.1.gz ln -s xxxxxxxxxxxxxxxxx etc/alternatives/ex ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/ex.1.gz ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/ex.fr.1.gz ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/ex.it.1.gz ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/ex.pl.1.gz ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/ex.ru.1.gz ln -s xxxxxxxxxxxxxxxxx etc/alternatives/from ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/from.1.gz ln -s xxxxxxxxxxxxxxxxxxx etc/alternatives/ftp ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/ftp.1.gz ln -s xxxxxxxxxxxxxxxxxxx etc/alternatives/ifenslave ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/ifenslave.8.gz ln -s xxxxxxxxxxxxx etc/alternatives/infobrowser ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/infobrowser.1.gz ln -s xxxxxxxxxxxxxxx etc/alternatives/lft ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/lft.1.gz ln -s xxxxxxxxxxxxxxxx etc/alternatives/locate ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/locate.1.gz ln -s xxxxxxxxxxxxxxxxxx etc/alternatives/mail ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/mail.1.gz ln -s xxxxxxxxxxxxxxxxxx etc/alternatives/mailx ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/mailx.1.gz ln -s xxxxxxxxxxx etc/alternatives/mt ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/mt.1.gz ln -s xxxxxxxxxxxxxxxxx etc/alternatives/mutt ln -s xxxxxxxxxxxxx etc/alternatives/nawk ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/nawk.1.gz ln -s xxxxxxxxxxxxxxxxxxx etc/alternatives/nc ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/nc.1.gz ln -s xxxxxxxxxxxxxxxxxxx etc/alternatives/netcat ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/netcat.1.gz ln -s xxxxxxxxx etc/alternatives/pager ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/pager.1.gz ln -s xxxxxxxxx etc/alternatives/pico ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/pico.1.gz ln -s xxxxxxxxxxxx etc/alternatives/rcp ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/rcp.1.gz ln -s xxxxxxxxxxxxxxxx etc/alternatives/rename ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/rename.1.gz ln -s xxxxxxxxxxxxxxx etc/alternatives/rlogin ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/rlogin.1.gz ln -s xxxxxxxxxxxxxxxxx etc/alternatives/rmt ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/rmt.8.gz ln -s xxxxxxxxxxxx etc/alternatives/rsh ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/rsh.1.gz ln -s xxxxxxxxxxxxxxxxx etc/alternatives/rview ln -s xxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/tcptraceroute ln -s xxxxxxxxxxxxxxxxxxxxxx etc/alternatives/telnet ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/tcptraceroute.8.gz ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/telnet.1.gz ln -s xxxxxxxxxxxxxxxxxxxxxx etc/alternatives/traceproto ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/traceproto.1.gz ln -s xxxxxxxxxxxxxxxxxxxxxx etc/alternatives/traceroute ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/traceroute.1.gz ln -s xxxxxxxxxxxxxxxxxxxxxx etc/alternatives/traceroute.sbin ln -s xxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/traceroute6 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/traceroute6.1.gz ln -s xxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/updatedb ln -s xxxxxxxxxxxxxxxxx etc/alternatives/vi ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/vi.1.gz ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/vi.fr.1.gz ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/vi.it.1.gz ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/vi.pl.1.gz ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/vi.ru.1.gz ln -s xxxxxxxxxxxxxxxxx etc/alternatives/view ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/view.1.gz ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/view.fr.1.gz ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/view.it.1.gz ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/view.pl.1.gz ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/view.ru.1.gz ln -s xxxxxxxxxxxxxxxxx etc/alternatives/w ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/w.1.gz ln -s xxxxxxxxxxxxxxxxxx etc/alternatives/write ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/write.1.gz ln -s xxxxxxxxxxxx etc/alternatives/www-browser ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/alternatives/www-browser.1.gz ln -s xxxxxx etc/dpkg/origins/default ln -s xxxxxxxxxxxxx etc/rc0.d/K01atd ln -s xxxxxxxxxxxxxxx etc/rc0.d/K01exim4 ln -s xxxxxxxxxxxxxxxxxxx etc/rc0.d/K01heartbeat ln -s xxxxxxxxxxxxxx etc/rc0.d/K01logd ln -s xxxxxxxxxxxxxxxxxxxxx etc/rc0.d/K01mysql-proxy ln -s xxxxxxxxxxxxxxxxx etc/rc0.d/K01urandom ln -s xxxxxxxxxxxxxxxxxxxxxx etc/rc0.d/K02mysql.server ln -s xxxxxxxxxxxxxxxxxx etc/rc0.d/K02openhpid ln -s xxxxxxxxxxxxxxxxxx etc/rc0.d/K03sendsigs ln -s xxxxxxxxxxxxxxxxx etc/rc0.d/K04rsyslog ln -s xxxxxxxxxxxxxxxxxxxxxx etc/rc0.d/K05umountnfs.sh ln -s xxxxxxxxxxxxxxxxxxxx etc/rc0.d/K06nfs-common ln -s xxxxxxxxxxxxxxxxx etc/rc0.d/K06portmap ln -s xxxxxxxxxxxxxxxxxxxx etc/rc0.d/K07hwclock.sh ln -s xxxxxxxxxxxxxxxxxxxx etc/rc0.d/K07networking ln -s xxxxxxxxxxxxxxxxxx etc/rc0.d/K08ifupdown ln -s xxxxxxxxxxxxxxxxxx etc/rc0.d/K09umountfs ln -s xxxxxxxxxxxxxx etc/rc0.d/K10lvm2 ln -s xxxxxxxxxxxxxxxxxxxx etc/rc0.d/K11umountroot ln -s xxxxxxxxxxxxxx etc/rc0.d/K12halt ln -s xxxxxxxxxxxxx etc/rc1.d/K01atd ln -s xxxxxxxxxxxxxxx etc/rc1.d/K01exim4 ln -s xxxxxxxxxxxxxxxxxxx etc/rc1.d/K01heartbeat ln -s xxxxxxxxxxxxxx etc/rc1.d/K01logd ln -s xxxxxxxxxxxxxxxxxxxxx etc/rc1.d/K01mysql-proxy ln -s xxxxxxxxxxxxxxxxxxxxxx etc/rc1.d/K02mysql.server ln -s xxxxxxxxxxxxxxxxxx etc/rc1.d/K02openhpid ln -s xxxxxxxxxxxxxxxxx etc/rc1.d/K04rsyslog ln -s xxxxxxxxxxxxxxxxxxxx etc/rc1.d/K06nfs-common ln -s xxxxxxxxxxxxxxxxx etc/rc1.d/K06portmap ln -s xxxxxxxxxxxxxxxxxxx etc/rc1.d/S01killprocs ln -s xxxxxxxxxxxxxxxxxx etc/rc1.d/S20bootlogs ln -s xxxxxxxxxxxxxxxx etc/rc1.d/S21single ln -s xxxxxxxxxxxxxxxxx etc/rc2.d/S14portmap ln -s xxxxxxxxxxxxxxxxxxxx etc/rc2.d/S15nfs-common ln -s xxxxxxxxxxxxxxxxxxxx etc/rc2.d/S17fancontrol ln -s xxxxxxxxxxxxxxxxxxxxxx etc/rc2.d/S17mysql.server ln -s xxxxxxxxxxxxxxxxx etc/rc2.d/S17rsyslog ln -s xxxxxxxxxxxxxxx etc/rc2.d/S18acpid ln -s xxxxxxxxxxxxx etc/rc2.d/S18atd ln -s xxxxxxxxxxxxxx etc/rc2.d/S18cron ln -s xxxxxxxxxxxxxxx etc/rc2.d/S18exim4 ln -s xxxxxxxxxxxxxx etc/rc2.d/S18logd ln -s xxxxxxxxxxxxxxxxxxxxx etc/rc2.d/S18mysql-proxy ln -s xxxxxxxxxxxxxxxxxx etc/rc2.d/S18openhpid ln -s xxxxxxxxxxxxxxx etc/rc2.d/S18rsync ln -s xxxxxxxxxxxxx etc/rc2.d/S18ssh ln -s xxxxxxxxxxxxxxxxxxx etc/rc2.d/S19heartbeat ln -s xxxxxxxxxxxxxxxxxx etc/rc2.d/S20bootlogs ln -s xxxxxxxxxxxxxxxxxx etc/rc2.d/S21rc.local ln -s xxxxxxxxxxxxxxxxxxx etc/rc2.d/S21rmnologin ln -s xxxxxxxxxxxxxxxxxxxxxxx etc/rc2.d/S21stop-bootlogd ln -s xxxxxxxxxxxxxxxxx etc/rc3.d/S14portmap ln -s xxxxxxxxxxxxxxxxxxxx etc/rc3.d/S15nfs-common ln -s xxxxxxxxxxxxxxxxxxxx etc/rc3.d/S17fancontrol ln -s xxxxxxxxxxxxxxxxxxxxxx etc/rc3.d/S17mysql.server ln -s xxxxxxxxxxxxxxxxx etc/rc3.d/S17rsyslog ln -s xxxxxxxxxxxxxxx etc/rc3.d/S18acpid ln -s xxxxxxxxxxxxx etc/rc3.d/S18atd ln -s xxxxxxxxxxxxxx etc/rc3.d/S18cron ln -s xxxxxxxxxxxxxxx etc/rc3.d/S18exim4 ln -s xxxxxxxxxxxxxx etc/rc3.d/S18logd ln -s xxxxxxxxxxxxxxxxxxxxx etc/rc3.d/S18mysql-proxy ln -s xxxxxxxxxxxxxxxxxx etc/rc3.d/S18openhpid ln -s xxxxxxxxxxxxxxx etc/rc3.d/S18rsync ln -s xxxxxxxxxxxxx etc/rc3.d/S18ssh ln -s xxxxxxxxxxxxxxxxxxx etc/rc3.d/S19heartbeat ln -s xxxxxxxxxxxxxxxxxx etc/rc3.d/S20bootlogs ln -s xxxxxxxxxxxxxxxxxx etc/rc3.d/S21rc.local ln -s xxxxxxxxxxxxxxxxxxx etc/rc3.d/S21rmnologin ln -s xxxxxxxxxxxxxxxxxxxxxxx etc/rc3.d/S21stop-bootlogd ln -s xxxxxxxxxxxxxxxxx etc/rc4.d/S14portmap ln -s xxxxxxxxxxxxxxxxxxxx etc/rc4.d/S15nfs-common ln -s xxxxxxxxxxxxxxxxxxxx etc/rc4.d/S17fancontrol ln -s xxxxxxxxxxxxxxxxxxxxxx etc/rc4.d/S17mysql.server ln -s xxxxxxxxxxxxxxxxx etc/rc4.d/S17rsyslog ln -s xxxxxxxxxxxxxxx etc/rc4.d/S18acpid ln -s xxxxxxxxxxxxx etc/rc4.d/S18atd ln -s xxxxxxxxxxxxxx etc/rc4.d/S18cron ln -s xxxxxxxxxxxxxxx etc/rc4.d/S18exim4 ln -s xxxxxxxxxxxxxx etc/rc4.d/S18logd ln -s xxxxxxxxxxxxxxxxxxxxx etc/rc4.d/S18mysql-proxy ln -s xxxxxxxxxxxxxxxxxx etc/rc4.d/S18openhpid ln -s xxxxxxxxxxxxxxx etc/rc4.d/S18rsync ln -s xxxxxxxxxxxxx etc/rc4.d/S18ssh ln -s xxxxxxxxxxxxxxxxxxx etc/rc4.d/S19heartbeat ln -s xxxxxxxxxxxxxxxxxx etc/rc4.d/S20bootlogs ln -s xxxxxxxxxxxxxxxxxx etc/rc4.d/S21rc.local ln -s xxxxxxxxxxxxxxxxxxx etc/rc4.d/S21rmnologin ln -s xxxxxxxxxxxxxxxxxxxxxxx etc/rc4.d/S21stop-bootlogd ln -s xxxxxxxxxxxxxxxxx etc/rc5.d/S14portmap ln -s xxxxxxxxxxxxxxxxxxxx etc/rc5.d/S15nfs-common ln -s xxxxxxxxxxxxxxxxxxxx etc/rc5.d/S17fancontrol ln -s xxxxxxxxxxxxxxxxxxxxxx etc/rc5.d/S17mysql.server ln -s xxxxxxxxxxxxxxxxx etc/rc5.d/S17rsyslog ln -s xxxxxxxxxxxxxxx etc/rc5.d/S18acpid ln -s xxxxxxxxxxxxx etc/rc5.d/S18atd ln -s xxxxxxxxxxxxxx etc/rc5.d/S18cron ln -s xxxxxxxxxxxxxxx etc/rc5.d/S18exim4 ln -s xxxxxxxxxxxxxx etc/rc5.d/S18logd ln -s xxxxxxxxxxxxxxxxxxxxx etc/rc5.d/S18mysql-proxy ln -s xxxxxxxxxxxxxxxxxx etc/rc5.d/S18openhpid ln -s xxxxxxxxxxxxxxx etc/rc5.d/S18rsync ln -s xxxxxxxxxxxxx etc/rc5.d/S18ssh ln -s xxxxxxxxxxxxxxxxxxx etc/rc5.d/S19heartbeat ln -s xxxxxxxxxxxxxxxxxx etc/rc5.d/S20bootlogs ln -s xxxxxxxxxxxxxxxxxx etc/rc5.d/S21rc.local ln -s xxxxxxxxxxxxxxxxxxx etc/rc5.d/S21rmnologin ln -s xxxxxxxxxxxxxxxxxxxxxxx etc/rc5.d/S21stop-bootlogd ln -s xxxxxxxxxxxxx etc/rc6.d/K01atd ln -s xxxxxxxxxxxxxxx etc/rc6.d/K01exim4 ln -s xxxxxxxxxxxxxxxxxxx etc/rc6.d/K01heartbeat ln -s xxxxxxxxxxxxxx etc/rc6.d/K01logd ln -s xxxxxxxxxxxxxxxxxxxxx etc/rc6.d/K01mysql-proxy ln -s xxxxxxxxxxxxxxxxx etc/rc6.d/K01urandom ln -s xxxxxxxxxxxxxxxxxxxxxx etc/rc6.d/K02mysql.server ln -s xxxxxxxxxxxxxxxxxx etc/rc6.d/K02openhpid ln -s xxxxxxxxxxxxxxxxxx etc/rc6.d/K03sendsigs ln -s xxxxxxxxxxxxxxxxx etc/rc6.d/K04rsyslog ln -s xxxxxxxxxxxxxxxxxxxxxx etc/rc6.d/K05umountnfs.sh ln -s xxxxxxxxxxxxxxxxxxxx etc/rc6.d/K06nfs-common ln -s xxxxxxxxxxxxxxxxx etc/rc6.d/K06portmap ln -s xxxxxxxxxxxxxxxxxxxx etc/rc6.d/K07hwclock.sh ln -s xxxxxxxxxxxxxxxxxxxx etc/rc6.d/K07networking ln -s xxxxxxxxxxxxxxxxxx etc/rc6.d/K08ifupdown ln -s xxxxxxxxxxxxxxxxxx etc/rc6.d/K09umountfs ln -s xxxxxxxxxxxxxx etc/rc6.d/K10lvm2 ln -s xxxxxxxxxxxxxxxxxxxx etc/rc6.d/K11umountroot ln -s xxxxxxxxxxxxxxxx etc/rc6.d/K12reboot ln -s xxxxxxxxxxxxxxxxxxxxxxxx etc/rcS.d/S01mountkernfs.sh ln -s xxxxxxxxxxxxxx etc/rcS.d/S02udev ln -s xxxxxxxxxxxxxxxxxxxxxxxxxx etc/rcS.d/S03mountdevsubfs.sh ln -s xxxxxxxxxxxxxxxxxx etc/rcS.d/S04bootlogd ln -s xxxxxxxxxxxxxxxxxxxxxxxx etc/rcS.d/S05keyboard-setup ln -s xxxxxxxxxxxxxxxxxxxxx etc/rcS.d/S06hostname.sh ln -s xxxxxxxxxxxxxxxxxxxxxxxxx etc/rcS.d/S06hwclockfirst.sh ln -s xxxxxxxxxxxxxx etc/rcS.d/S06lvm2 ln -s xxxxxxxxxxxxxxxxxxxxxx etc/rcS.d/S07checkroot.sh ln -s xxxxxxxxxxxxxxxxxxxx etc/rcS.d/S08hwclock.sh ln -s xxxxxxxxxxxxxxxxxxxxxxxx etc/rcS.d/S08ifupdown-clean ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxx etc/rcS.d/S08module-init-tools ln -s xxxxxxxxxxxxxxxxx etc/rcS.d/S08mtab.sh ln -s xxxxxxxxxxxxxxxxxxxx etc/rcS.d/S09checkfs.sh ln -s xxxxxxxxxxxxxxxxxx etc/rcS.d/S10ifupdown ln -s xxxxxxxxxxxxxxxxxxxxx etc/rcS.d/S10mountall.sh ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/rcS.d/S11mountall-bootclean.sh ln -s xxxxxxxxxxxxxxxxxxxxxxxxxx etc/rcS.d/S12mountoverflowtmp ln -s xxxxxxxxxxxxxxxxxxxx etc/rcS.d/S13networking ln -s xxxxxxxxxxxxx etc/rcS.d/S18kbd ln -s xxxxxxxxxxxxxxxx etc/rcS.d/S13procps ln -s xxxxxxxxxxxxxxxxxxx etc/rcS.d/S13udev-mtab ln -s xxxxxxxxxxxxxxxxx etc/rcS.d/S13urandom ln -s xxxxxxxxxxxxxxxxx etc/rcS.d/S14portmap ln -s xxxxxxxxxxxxxxxxxxxx etc/rcS.d/S15nfs-common ln -s xxxxxxxxxxxxxxxxxxxxx etc/rcS.d/S16mountnfs.sh ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/rcS.d/S17mountnfs-bootclean.sh ln -s xxxxxxxxxxxxxxxxxxxxxxx etc/rcS.d/S19console-setup ln -s xxxxxxxxxxxxxxxxxxxxx etc/rcS.d/S20bootmisc.sh ln -s xxxxxxxxxxxxxxxxxxxx etc/rcS.d/S20lm-sensors ln -s xxxxxxxxxxxxxxxxxxxxxxxx etc/rcS.d/S20screen-cleanup ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/rcS.d/S21stop-bootlogd-single ln -s xxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/00673b5b.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/02b73561.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/0481cb65.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/052e396b.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/06d75f4b.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/08aef7bb.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/0c364b2d.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/0dbd0096.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/0e82f83a.0 ln -s xxxxxxxxxxxxxxxxxx etc/ssl/certs/10d149a2.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/111e6273.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/1155c94b.0 ln -s xxxxxxxxxxxxxxxxxxxx etc/ssl/certs/11a09b38.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/11f154d6.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/124bbd54.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/128b9c8d.0 ln -s xxxxxxxxxxxxxxxxxx etc/ssl/certs/12d55845.0 ln -s xxxxxxxxxxxxxxxxxx etc/ssl/certs/1689a10b.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/17b51fe6.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/19899da5.0 ln -s xxxxxxxxxxxxxxxxxx etc/ssl/certs/1dac3003.0 ln -s xxxxxxxxxxxxxxx etc/ssl/certs/1dcd6f4c.0 ln -s xxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/219d9499.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/23f4c490.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/256fd83b.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/2edf7016.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/2fa87019.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/2fb1850a.0 ln -s xxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/31044350.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/33815e15.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/399e7759.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/3ad48a91.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/3c58f906.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/3c860d51.0 ln -s xxxxxxxxxxxxxxxxxxx etc/ssl/certs/3e223c08.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/3e7271e8.0 ln -s xxxxxxxxxxxxxxx etc/ssl/certs/408e388a.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/412bea73.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/4166ec0c.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/4184de39.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/4643210f.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/46b2fd3b.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/47996b5c.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/4d654d1d.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/4e18c148.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/4fbd6bfa.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/5046c355.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/54edfa5d.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/56b8a0b6.0 ln -s xxxxxxxxxxxxxxxxxxx etc/ssl/certs/56e29e75.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/57692373.0 ln -s xxxxxxxxxxxxxxxxx etc/ssl/certs/58a44af1.0 ln -s xxxxxxxxxxxxxxxxx etc/ssl/certs/58a44af1.1 ln -s xxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/594f1775.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/5a3f0ff8.0 ln -s xxxxxxxxxxxxxxxxx=xxxxxxx=xxxxxxxxx etc/ssl/certs/5a5372fc.0 ln -s xxxxxxxxxxxxxxxxxxxx etc/ssl/certs/5cf9d536.0 ln -s xxxxxxxxxxxxxx etc/ssl/certs/5ed36f99.0 ln -s xxxxxxxxxxxxxxxx=xxxxxxx=xxxxxxxxx etc/ssl/certs/635ccfd5.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/69105f4f.0 ln -s xxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/6adf0799.0 ln -s xxxxxx etc/ssl/certs/6e52cc39.0 ln -s xxxxxxxxxxxxxxxxxx etc/ssl/certs/6e8bf996.0 ln -s xxxxxxxxxxxxxxxxx etc/ssl/certs/6f5d9899.0 ln -s xxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/6fcc125d.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/709afd2b.0 ln -s xxxxxxxxxxxxxxxxxxxx etc/ssl/certs/72bf6a04.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/72fa7371.0 ln -s xxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/731f03a5.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/74c26bd0.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/755f7420.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/75680d2e.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/7651b327.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/76579174.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/788c9bfc.0 ln -s xxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/7999be0d.0 ln -s xxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/7a819ef2.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/7d3cd826.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/7d453d8f.0 ln -s xxxxxxxxxxxxxxxxxxx etc/ssl/certs/819a45f6.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/81b9768f.0 ln -s xxxxxxxxxxxxxxxxxx=xxxxxxxx=xxxxxxxxx etc/ssl/certs/8317b10c.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/8470719d.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/84cba82f.0 ln -s xxxxxxxxxxxxxxxxxxx etc/ssl/certs/86f32474.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/87753b0d.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/89c02a45.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/8f7b96c4.0 ln -s xxxxxxxxxxxxxxxxxxxx etc/ssl/certs/8fe643df.0 ln -s xxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/9339512a.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/95750816.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/9772ca32.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/97b4211c.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/9ec3a561.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=xxxxxxxxxxxxxxxxxxxxxxx=xxxxxxxxxxxx etc/ssl/certs/ABAecom_=sub.__Am._Bankers_Assn.=_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/AddTrust_External_Root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/a0bc6fbb.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/AOL_Time_Warner_Root_Certification_Authority_1.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/AOL_Time_Warner_Root_Certification_Authority_2.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/AddTrust_Low-Value_Services_Root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/AddTrust_Public_Services_Root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/AddTrust_Qualified_Certificates_Root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/America_Online_Root_Certification_Authority_1.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/America_Online_Root_Certification_Authority_2.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Baltimore_CyberTrust_Root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/COMODO_Certification_Authority.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Certplus_Class_2_Primary_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/COMODO_ECC_Certification_Authority.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Camerfirma_Chambers_of_Commerce_Root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Camerfirma_Global_Chambersign_Root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Certum_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Comodo_AAA_Services_root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Comodo_Secure_Services_root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Comodo_Trusted_Services_root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/DST_ACES_CA_X6.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/DST_Root_CA_X3.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/DigiCert_Assured_ID_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/DigiCert_Global_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/DigiCert_High_Assurance_EV_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Equifax_Secure_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Digital_Signature_Trust_Co._Global_CA_1.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Digital_Signature_Trust_Co._Global_CA_2.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Digital_Signature_Trust_Co._Global_CA_3.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Digital_Signature_Trust_Co._Global_CA_4.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Entrust.net_Global_Secure_Personal_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Entrust.net_Global_Secure_Server_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Entrust.net_Premium_2048_Secure_Server_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Entrust.net_Secure_Personal_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Entrust.net_Secure_Server_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Entrust_Root_Certification_Authority.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Equifax_Secure_Global_eBusiness_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Equifax_Secure_eBusiness_CA_1.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Equifax_Secure_eBusiness_CA_2.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Firmaprofesional_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/GTE_CyberTrust_Global_Root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/GTE_CyberTrust_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/GeoTrust_Global_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/GeoTrust_Global_CA_2.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/GeoTrust_Primary_Certification_Authority.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/GeoTrust_Universal_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/GeoTrust_Universal_CA_2.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/GlobalSign_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Go_Daddy_Class_2_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/GlobalSign_Root_CA_-_R2.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/IPS_CLASE1_root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/IPS_CLASE3_root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/IPS_CLASEA1_root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/IPS_CLASEA3_root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/IPS_Chained_CAs_root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/IPS_Servidores_root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/IPS_Timestamping_root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/NetLock_Business_=Class_B=_Root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/NetLock_Express_=Class_C=_Root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/NetLock_Notary_=Class_A=_Root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/NetLock_Qualified_=Class_QA=_Root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Network_Solutions_Certificate_Authority.pem ln -s xxxxxxxxxxxxxxxxxxx etc/ssl/certs/a145806c.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/QuoVadis_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/QuoVadis_Root_CA_2.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/QuoVadis_Root_CA_3.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/RSA_Root_Certificate_1.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/RSA_Security_1024_v3.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/RSA_Security_2048_v3.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/SecureTrust_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Secure_Global_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Security_Communication_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Sonera_Class_1_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Sonera_Class_2_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Staat_der_Nederlanden_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Starfield_Class_2_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/StartCom_Certification_Authority.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/StartCom_Ltd..pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/SwissSign_Gold_CA_-_G2.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/SwissSign_Platinum_CA_-_G2.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/SwissSign_Silver_CA_-_G2.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Swisscom_Root_CA_1.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/TC_TrustCenter__Germany__Class_2_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/TC_TrustCenter__Germany__Class_3_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/TDC_Internet_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/TDC_OCES_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/TURKTRUST_Certificate_Services_Provider_Root_1.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/TURKTRUST_Certificate_Services_Provider_Root_2.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Taiwan_GRCA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/a15b3b6b.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Thawte_Personal_Basic_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Thawte_Personal_Freemail_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Thawte_Personal_Premium_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Thawte_Premium_Server_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Thawte_Server_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Thawte_Time_Stamping_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/UTN-USER_First-Network_Applications.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/UTN_DATACorp_SGC_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/UTN_USERFirst_Email_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/UTN_USERFirst_Hardware_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/ValiCert_Class_1_VA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/ValiCert_Class_2_VA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Verisign_RSA_Secure_Server_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/VeriSign_Class_3_Public_Primary_Certification_Authority_-_G5.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Verisign_Class_1_Public_Primary_Certification_Authority.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Verisign_Class_1_Public_Primary_Certification_Authority_-_G2.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Verisign_Class_1_Public_Primary_Certification_Authority_-_G3.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Verisign_Class_2_Public_Primary_Certification_Authority.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Verisign_Class_2_Public_Primary_Certification_Authority_-_G2.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Verisign_Class_3_Public_Primary_Certification_Authority.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Verisign_Class_2_Public_Primary_Certification_Authority_-_G3.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Verisign_Class_3_Public_Primary_Certification_Authority_-_G2.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Verisign_Class_3_Public_Primary_Certification_Authority_-_G3.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Verisign_Class_4_Public_Primary_Certification_Authority_-_G2.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Verisign_Class_4_Public_Primary_Certification_Authority_-_G3.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Verisign_Time_Stamping_Authority_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Visa_International_Global_Root_2.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Visa_eCommerce_Root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/WellsSecure_Public_Root_Certificate_Authority.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/Wells_Fargo_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/XRamp_Global_CA_Root.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/a2df7ad7.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/a3896b44.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/a6776c69.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/a7605362.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/aaa45464.0 ln -s xxxxxxxxxxxxxxx=xxxxxxx=xxxxxxxxx etc/ssl/certs/add67345.0 ln -s xxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/b0f3e76e.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/b3fec4ff.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/b5f329fa.0 ln -s xxxxxxxxxxxxxxxxxx etc/ssl/certs/b8609e8a.0 ln -s xxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/bcdd5959.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/bda4cc84.0 ln -s xxxxxxxxxxxxxxxxxxxx etc/ssl/certs/bdacca6f.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/beTRUSTed_Root_CA-Baltimore_Implementation.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/beTRUSTed_Root_CA.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/bf64f35b.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/ca.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/beTRUSTed_Root_CA_-_Entrust_Implementation.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/beTRUSTed_Root_CA_-_RSA_Implementation.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/bf87590f.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/brasil.gov.br.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/c19d42c7.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/c215bc69.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/c33a80d4.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/c527e4ab.0 ln -s xxxxxxxx=xxxxxxxxxxxxxxxxxxxxxxx=xxxxxxxxxxxx etc/ssl/certs/c9bc75ba.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/cacert.org.pem ln -s xxxxxxxxxxxxxxxxx etc/ssl/certs/cb796bc1.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/cdaebb72.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/cert_igca_dsa.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/cert_igca_rsa.pem ln -s xxxxxxxxxxxxxxxxxx etc/ssl/certs/cf701eeb.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/d2adc77d.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/d537fba6.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/d78a75c7.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/d8274e24.0 ln -s xxxxxxxxxxxxxxxxxxxx etc/ssl/certs/ddc328ff.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/deutsche-telekom-root-ca-2.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/e268a4c5.0 ln -s xxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/e60bf0c0.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/e775ed2d.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/e7b8d656.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/ed049835.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/ed524cf5.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/ed62f4e3.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/ee7cd6fb.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/f3cf1e8e.0 ln -s xxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/f4996e82.0 ln -s xxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/f64d9715.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/f73e89fd.0 ln -s xxxxxxxxxxxxxxxxxxx etc/ssl/certs/f950ccc2.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/facacbc6.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/ff783690.0 ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/signet_ca1_pem.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/signet_ca2_pem.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/signet_ca3_pem.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/signet_ocspklasa2_pem.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/signet_ocspklasa3_pem.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/signet_pca2_pem.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/signet_pca3_pem.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/signet_rootca_pem.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/signet_tsa1_pem.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/spi-ca-2003.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/spi-cacert-2008.pem ln -s xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx etc/ssl/certs/thawte_Primary_Root_CA.pem find . -exec setextattr -h user t "XXXXXXXXXXXXXX" \{} \; --J/dobhs11T7y2rNN-- From owner-freebsd-fs@FreeBSD.ORG Tue Sep 11 16:58:14 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C8FCC106566B for ; Tue, 11 Sep 2012 16:58:14 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 21C118FC17 for ; Tue, 11 Sep 2012 16:58:13 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA26712; Tue, 11 Sep 2012 19:58:05 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <504F6D9C.2040000@FreeBSD.org> Date: Tue, 11 Sep 2012 19:58:04 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120830 Thunderbird/15.0 MIME-Version: 1.0 To: Nicolas Rachinsky References: <20120911161049.GA21635@mid.pc5.i.0x5.de> In-Reply-To: <20120911161049.GA21635@mid.pc5.i.0x5.de> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org Subject: Re: bug? with zfs, symlinks and extended attributes X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 16:58:14 -0000 on 11/09/2012 19:10 Nicolas Rachinsky said the following: > Hallo, > > I can reliable "break" zfs on FreeBSD 8.3-RELEASE-p4 (on two different > machines). Please check if http://svn.freebsd.org/changeset/base/240345 may help you. I think that the diff should be applicable to your version. > I just do the following: > > zpool create pool1 ada0p15 > zfs create pool1/TST > cd /pool1/TST > sh attached script > cd - > zfs umount pool1/TST > zfs mount pool1/TST > > Afterwards some of the symlinks in /pool1/TST have extended attributes > I've never set. Others cannot have any extended attributes. > > For example: > # lsextattr -h user /pool1/TST/etc/rc0.d/K04rsyslog > lsextattr: /pool1/TST/etc/rc0.d/K04rsyslog: failed: Invalid argument > > setextattr fails as well. > > Not all symlinks show this problem. The set showing the problem seems > to be not always the same. > > With the following changes to the attached script, I cannot reproduce > the problem: > > leaving out all touch-lines. > > set all extended attributes to a string of length 1. > > let all symlinks point to foo. > > The problem does occur when I run the script as normal user. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Tue Sep 11 17:42:29 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DEE84106564A for ; Tue, 11 Sep 2012 17:42:29 +0000 (UTC) (envelope-from mike@bayphoto.com) Received: from mx.got.net (mx1.mx3.got.net [207.111.237.40]) by mx1.freebsd.org (Postfix) with ESMTP id C78F98FC0A for ; Tue, 11 Sep 2012 17:42:29 +0000 (UTC) Received: from [192.168.11.47] (delaware.bayphoto.com [207.111.245.164]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx.got.net (mx1.mx3.got.net) with ESMTP id 0509F14F8B for ; Tue, 11 Sep 2012 10:24:27 -0700 (PDT) Message-ID: <504F73CA.2090200@bayphoto.com> Date: Tue, 11 Sep 2012 10:24:26 -0700 From: Mike Carlson User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) Gecko/20120824 Thunderbird/15.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <504E627C.70408@theiconic.com.au> In-Reply-To: <504E627C.70408@theiconic.com.au> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: clustered file systems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: mike@bayphoto.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 17:42:30 -0000 On 9/10/2012 2:58 PM, Jurgen Weber wrote: > Guys > > I require a SAN solution for a mail server implementation I am > designing. The design I am looking for is the following: > > https://www.communigate.com/cgatepro/ClusterDynamic.html#OSCluster > > This means I require some sort of SAN storage managed by a clustered > FS, having very little SAN + ClusterFS experience I am research the > options available to me. FreeBSD being my OS of choice but I found the > following: > > http://blog.elitecoderz.net/cluster-filesystem-for-freebsd-gfs-ocfs2/2010/06/ > > > This blogger mentions that their is no Clustered FS support at this > point in time for FreeBSD which I find astonishing but I have been > unable to find anything except for a GlusterFS port using fuse (not > ideal). > > Any thoughts or suggestions appreciated. > > Thanks > MooseFS is in the ports tree ( sysutils/moosefs-master , sysutils/moosefs-chunkserver , sysutils/moosefs-client ) I tested it out on a few VM's nearly two years ago on FreeBSD 7 or 8, and it was as responsive as I could expect from three VM's running on the same desktop. If I had the resources and time, I would love to stress test it and see how it scales. I do agree that FreeBSD could really use a good clustered filesystem. I thought someone was working on CephFS at one point but I can no longer find any references to that. Mike C From owner-freebsd-fs@FreeBSD.ORG Tue Sep 11 19:14:54 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3412B106564A; Tue, 11 Sep 2012 19:14:54 +0000 (UTC) (envelope-from nicolas@i.0x5.de) Received: from n.0x5.de (n.0x5.de [217.197.85.144]) by mx1.freebsd.org (Postfix) with ESMTP id DF3108FC08; Tue, 11 Sep 2012 19:14:53 +0000 (UTC) Received: by pc5.i.0x5.de (Postfix, from userid 1003) id 3XGbRJ1tg9z7ySD; Tue, 11 Sep 2012 21:14:52 +0200 (CEST) Date: Tue, 11 Sep 2012 21:14:52 +0200 From: Nicolas Rachinsky To: Andriy Gapon Message-ID: <20120911191452.GA48029@mid.pc5.i.0x5.de> References: <20120911161049.GA21635@mid.pc5.i.0x5.de> <504F6D9C.2040000@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <504F6D9C.2040000@FreeBSD.org> X-Powered-by: FreeBSD X-Homepage: http://www.rachinsky.de X-PGP-Keyid: 887BAE72 X-PGP-Fingerprint: 039E 9433 115F BC5F F88D 4524 5092 45C4 887B AE72 X-PGP-Keys: http://www.rachinsky.de/nicolas/gpg/nicolas_rachinsky.asc User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@FreeBSD.org Subject: Re: bug? with zfs, symlinks and extended attributes X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 19:14:54 -0000 * Andriy Gapon [2012-09-11 19:58 +0300]: > on 11/09/2012 19:10 Nicolas Rachinsky said the following: > > I can reliable "break" zfs on FreeBSD 8.3-RELEASE-p4 (on two different > > machines). > > Please check if http://svn.freebsd.org/changeset/base/240345 may help you. > I think that the diff should be applicable to your version. The patch applies fine, it compiles, but it does not help. Thanks for your efforts Nicolas -- http://www.rachinsky.de/nicolas From owner-freebsd-fs@FreeBSD.ORG Tue Sep 11 19:52:33 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 54D27106566C for ; Tue, 11 Sep 2012 19:52:33 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A029C8FC14 for ; Tue, 11 Sep 2012 19:52:32 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id WAA28461; Tue, 11 Sep 2012 22:52:29 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TBWVZ-0002va-4y; Tue, 11 Sep 2012 22:52:29 +0300 Message-ID: <504F967B.6030108@FreeBSD.org> Date: Tue, 11 Sep 2012 22:52:27 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120901 Thunderbird/15.0 MIME-Version: 1.0 To: Nicolas Rachinsky References: <20120911161049.GA21635@mid.pc5.i.0x5.de> <504F6D9C.2040000@FreeBSD.org> <20120911191452.GA48029@mid.pc5.i.0x5.de> In-Reply-To: <20120911191452.GA48029@mid.pc5.i.0x5.de> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org Subject: Re: bug? with zfs, symlinks and extended attributes X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 19:52:33 -0000 on 11/09/2012 22:14 Nicolas Rachinsky said the following: > * Andriy Gapon [2012-09-11 19:58 +0300]: >> on 11/09/2012 19:10 Nicolas Rachinsky said the following: >>> I can reliable "break" zfs on FreeBSD 8.3-RELEASE-p4 (on two different >>> machines). >> >> Please check if http://svn.freebsd.org/changeset/base/240345 may help you. >> I think that the diff should be applicable to your version. > > The patch applies fine, it compiles, but it does not help. > > Thanks for your efforts In your script do all those "xxx..." actually mean verbatim "xxx..." or are they just replacements for some scrubbed content? -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Tue Sep 11 20:10:27 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6AD66106568D; Tue, 11 Sep 2012 20:10:27 +0000 (UTC) (envelope-from nicolas@i.0x5.de) Received: from n.0x5.de (n.0x5.de [217.197.85.144]) by mx1.freebsd.org (Postfix) with ESMTP id 203EE8FC18; Tue, 11 Sep 2012 20:10:26 +0000 (UTC) Received: by pc5.i.0x5.de (Postfix, from userid 1003) id 3XGcgQ1Wlpz7yTK; Tue, 11 Sep 2012 22:10:26 +0200 (CEST) Date: Tue, 11 Sep 2012 22:10:26 +0200 From: Nicolas Rachinsky To: Andriy Gapon Message-ID: <20120911201026.GA12674@mid.pc5.i.0x5.de> References: <20120911161049.GA21635@mid.pc5.i.0x5.de> <504F6D9C.2040000@FreeBSD.org> <20120911191452.GA48029@mid.pc5.i.0x5.de> <504F967B.6030108@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <504F967B.6030108@FreeBSD.org> X-Powered-by: FreeBSD X-Homepage: http://www.rachinsky.de X-PGP-Keyid: 887BAE72 X-PGP-Fingerprint: 039E 9433 115F BC5F F88D 4524 5092 45C4 887B AE72 X-PGP-Keys: http://www.rachinsky.de/nicolas/gpg/nicolas_rachinsky.asc User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@FreeBSD.org Subject: Re: bug? with zfs, symlinks and extended attributes X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 20:10:27 -0000 * Andriy Gapon [2012-09-11 22:52 +0300]: > on 11/09/2012 22:14 Nicolas Rachinsky said the following: > > * Andriy Gapon [2012-09-11 19:58 +0300]: > >> on 11/09/2012 19:10 Nicolas Rachinsky said the following: > >>> I can reliable "break" zfs on FreeBSD 8.3-RELEASE-p4 (on two different > >>> machines). > >> > >> Please check if http://svn.freebsd.org/changeset/base/240345 may help you. > >> I think that the diff should be applicable to your version. > > > > The patch applies fine, it compiles, but it does not help. > > > > Thanks for your efforts > > In your script do all those "xxx..." actually mean verbatim "xxx..." or are they > just replacements for some scrubbed content? Verbatim "xxx...". There was other content, but replacing that with "xxx..." did not change anything. I did not change the length; when I let every symlink point to "foo", the results changed, and I did not find broken symlinks. Nicolas -- http://www.rachinsky.de/nicolas From owner-freebsd-fs@FreeBSD.ORG Tue Sep 11 21:32:18 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD11B106564A for ; Tue, 11 Sep 2012 21:32:18 +0000 (UTC) (envelope-from jurgen.weber@theiconic.com.au) Received: from exprod6og116.obsmtp.com (exprod6og116.obsmtp.com [64.18.1.37]) by mx1.freebsd.org (Postfix) with SMTP id 3C32A8FC0A for ; Tue, 11 Sep 2012 21:32:18 +0000 (UTC) Received: from mail-oa0-f54.google.com ([209.85.219.54]) (using TLSv1) by exprod6ob116.postini.com ([64.18.5.12]) with SMTP ID DSNKUE+t4TltToaaMFYbb4NlMmX2ITUHpMmz@postini.com; Tue, 11 Sep 2012 14:32:18 PDT Received: by oagm1 with SMTP id m1so705568oag.13 for ; Tue, 11 Sep 2012 14:32:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding :x-gm-message-state; bh=j/QzcB0rrLCx4RQnz6JP1kdRkAe+Y2h++1nMYWOlQrQ=; b=KYP+oOgEQHcffIYuQHaP37Q4K/JWCbXPDZCMALH8B989Pg1IX7z8Hh6fWM6WinDL6x uuMUnZvyVkfvhWgY0lFqmopt2ODb+HdchkR0XGse4OHwXJzDt2TWj+yvBD1W6IaktLMM FjDOXhtgF3gcyIPZaVK1aomGvyqjpUvnUh5dNQ3Z/FNvwMoG8fMLiCo6/N/MF1yD92QL JfSO9278/aL4IC0CogDlRAB+7rLFHfnchxqIl3ECVJgGOMNw6QRcnJb7iIGfgJAlI78U JhfxUuzo3tz6iHmo8ioQGThILnDHvl7k+tXy5wIMP8rotUOPmVF2q1c9oMVwm2m4B8m2 mBkA== Received: by 10.60.11.227 with SMTP id t3mr20018400oeb.91.1347399136589; Tue, 11 Sep 2012 14:32:16 -0700 (PDT) Received: from [172.20.24.157] ([202.126.107.170]) by mx.google.com with ESMTPS id a20sm13949391oei.2.2012.09.11.14.32.14 (version=SSLv3 cipher=OTHER); Tue, 11 Sep 2012 14:32:15 -0700 (PDT) Message-ID: <504FADDD.7000106@theiconic.com.au> Date: Wed, 12 Sep 2012 07:32:13 +1000 From: Jurgen Weber User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120827 Thunderbird/15.0 MIME-Version: 1.0 To: Erik Scholtz References: <504E627C.70408@theiconic.com.au> <504F1A46.60509@argonsoft.de> In-Reply-To: <504F1A46.60509@argonsoft.de> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Gm-Message-State: ALoCoQnsefkkQHswwtc1fwcgwHuXftzgXkat7INOzun9s62lgoPIe3XReTaPQl9VCEQT0Vlqhww+ Cc: freebsd-fs@freebsd.org Subject: Re: clustered file systems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 21:32:18 -0000 Erik yeah, ok........ a shame. A call to a clustered FS in FreeBSD! Thanks Jurgen On 11/09/12 21:02, Erik Scholtz wrote: > Jurgen, > > I'm facing the same problem (that's why I wrote this article on my > blog), and the only solution was to migrate from FreeBSD to CentOS (my > choice - others are also possible). > > Up to now, there is no Cluster FS for FreeBSD and imho there isn't > even a project with this target active. That's why I'm thinking, that > the best OS in this world will die in future. Clustering features are > key features these days. > > If anyone has another idea or information, I'm also very interested in > this! > > Greetings, > Erik -- Jurgen Weber Systems Engineer IT Infrastructure Team Leader THE ICONIC | E jurgen.weber@theiconic.com.au | www.theiconic.com.au From owner-freebsd-fs@FreeBSD.ORG Wed Sep 12 05:47:06 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D7FD6106564A for ; Wed, 12 Sep 2012 05:47:06 +0000 (UTC) (envelope-from Newsletter@goelli.de) Received: from mo6-p05-ob.rzone.de (mo6-p05-ob.rzone.de [IPv6:2a01:238:20a:202:5305::1]) by mx1.freebsd.org (Postfix) with ESMTP id 6ADE68FC08 for ; Wed, 12 Sep 2012 05:47:06 +0000 (UTC) X-RZG-CLASS-ID: mo05 X-RZG-AUTH: :ImkTZkytb+s5KUDumTG4i0mGDH1K4fweaf9O+/5rQT5iu9PH4l7+wOsTkwRfiNE= Received: from goelliNotebook (pD9FE2383.dip.t-dialin.net [217.254.35.131]) by smtp.strato.de (josoe mo20) (RZmta 30.14 DYNA|AUTH) with ESMTPA id z00828o8C4NE8k ; Wed, 12 Sep 2012 07:47:05 +0200 (CEST) From: =?UTF-8?Q?Thomas_G=C3=B6llner_=28Newsletter=29?= To: "'Volodymyr Kostyrko'" References: <001a01cd900d$bcfcc870$36f65950$@goelli.de> <504F282D.8030808@gmail.com> In-Reply-To: <504F282D.8030808@gmail.com> Date: Wed, 12 Sep 2012 07:47:04 +0200 Message-ID: <000a01cd90aa$0a277310$1e765930$@goelli.de> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQJeoooVG2eNN70r/NZmey5KTcQ1/AH9GaMbllQaZgA= Content-Language: de Cc: freebsd-fs@freebsd.org Subject: AW: ZFS: Corrupted pool metadata after adding vdev to a pool - no opportunity to rescue data from healthy vdevs? Remove a vdev? Rewrite metadata? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Sep 2012 05:47:06 -0000 >Looks pretty bad, but you can try readonly mode with transaction = rollback and recheck: > > zpool import -f -R /GoelliZFS1 -N -o readonly=3Don -F -X = GoelliZFS1 Thanks for your suggestion. I think it was a bit like "zpool import = -fFXa" so I'm not surprised that it has the same result after ca. 30 = seconds: cannot import 'GoelliZFS1': one or more devices is currently = unavailable If I can't tell the ZFS to ignore or delete the missing vdev, perhaps I = can replace the missing with a "new" one? I have the three 1.5TB discs I = wanted to add to the pool. So I can create a raidz-vdev out of them and = replace the missing-vdev with the new-raidz-vdev... Can someone tell me = how I am supposed to do this via CLI? Another idea is this: If I delete the pool, is there a way to take the = three 3TB discs with my data and create a "new-old" vdev/pool out of = them? I think of this because of the constructed case I mentioned... I still can't accept, that there's no way to rewrite or edit the = metadata ;-) From owner-freebsd-fs@FreeBSD.ORG Wed Sep 12 07:27:04 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5DAC31065673 for ; Wed, 12 Sep 2012 07:27:04 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id AA2128FC08 for ; Wed, 12 Sep 2012 07:27:03 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id KAA05193; Wed, 12 Sep 2012 10:27:00 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TBhLg-0006ER-2Q; Wed, 12 Sep 2012 10:27:00 +0300 Message-ID: <50503941.3030202@FreeBSD.org> Date: Wed, 12 Sep 2012 10:26:57 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120901 Thunderbird/15.0 MIME-Version: 1.0 To: Nicolas Rachinsky References: <20120911161049.GA21635@mid.pc5.i.0x5.de> <504F6D9C.2040000@FreeBSD.org> <20120911191452.GA48029@mid.pc5.i.0x5.de> <504F967B.6030108@FreeBSD.org> <20120911201026.GA12674@mid.pc5.i.0x5.de> In-Reply-To: <20120911201026.GA12674@mid.pc5.i.0x5.de> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org Subject: Re: bug? with zfs, symlinks and extended attributes X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Sep 2012 07:27:04 -0000 on 11/09/2012 23:10 Nicolas Rachinsky said the following: > Verbatim "xxx...". > > There was other content, but replacing that with "xxx..." did not > change anything. I did not change the length; when I let every > symlink point to "foo", the results changed, and I did not find > broken symlinks. Thank you for a very good testcase. Please try this patch. diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c index 9bb5cbc..a89f3be 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c @@ -578,7 +578,7 @@ sa_find_sizes(sa_os_t *sa, sa_bulk_attr_t *attr_desc, int attr_count, for (i = 0; i != attr_count; i++) { boolean_t is_var_sz; - *total += attr_desc[i].sa_length; + *total += P2ROUNDUP(attr_desc[i].sa_length, 8); if (done) goto next; -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Wed Sep 12 08:21:53 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9A95A106566C for ; Wed, 12 Sep 2012 08:21:53 +0000 (UTC) (envelope-from c.kworr@gmail.com) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 26D048FC0A for ; Wed, 12 Sep 2012 08:21:52 +0000 (UTC) Received: by eeke52 with SMTP id e52so1102865eek.13 for ; Wed, 12 Sep 2012 01:21:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=r+RsMMtbJLyZq7kSnIf0bbP8qbliUMcTx4gAiVzn3xY=; b=TDiuke4QbHK1OkKT910glO6+v6LjqvRQmU9xTcI7ntJQF0JCcMfqgkDFBVzhOlTZYt oWUvzBs5t4L52/jZY+AcWWmX7gT+yJUqEu9WTLUHLmgfbjmg+6jSO+qP3XnwmMJG+mMF vfnMFsZnI6PvtIYd82FolJzcnCZdcTUH5KapX6/h/Mi2Spqh9hISNL3kLQysgdM304Pe FP6tRIYGDLblu21h7t7p8txDksFQyoWXGI0z/87RXBMPhk2KDpb0mpuvilx3sO6mfjSW 7K/stG32eQwi7sL74qUcwC8KF4WIM2OWTi5/fxlpc5pGXR3X4qpSgg3VtfrkkVrxFSnu hMyQ== Received: by 10.204.156.10 with SMTP id u10mr5470624bkw.11.1347438111081; Wed, 12 Sep 2012 01:21:51 -0700 (PDT) Received: from green.local (186-102-132-95.pool.ukrtel.net. [95.132.102.186]) by mx.google.com with ESMTPS id n17sm10998731bks.6.2012.09.12.01.21.48 (version=SSLv3 cipher=OTHER); Wed, 12 Sep 2012 01:21:49 -0700 (PDT) Message-ID: <5050461A.9050608@gmail.com> Date: Wed, 12 Sep 2012 11:21:46 +0300 From: Volodymyr Kostyrko User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120911 Thunderbird/15.0.1 MIME-Version: 1.0 To: =?UTF-8?B?IlRob21hcyBHw7ZsbG5lciAoTmV3c2xldHRlciki?= References: <001a01cd900d$bcfcc870$36f65950$@goelli.de> <504F282D.8030808@gmail.com> <000a01cd90aa$0a277310$1e765930$@goelli.de> In-Reply-To: <000a01cd90aa$0a277310$1e765930$@goelli.de> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-fs@freebsd.org Subject: Re: AW: ZFS: Corrupted pool metadata after adding vdev to a pool - no opportunity to rescue data from healthy vdevs? Remove a vdev? Rewrite metadata? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Sep 2012 08:21:53 -0000 12.09.2012 08:47, Thomas Göllner (Newsletter) wrote: >> Looks pretty bad, but you can try readonly mode with transaction rollback and recheck: >> >> zpool import -f -R /GoelliZFS1 -N -o readonly=on -F -X GoelliZFS1 > > Thanks for your suggestion. I think it was a bit like "zpool import -fFXa" so I'm not surprised that it has the same result after ca. 30 seconds: > cannot import 'GoelliZFS1': one or more devices is currently unavailable You can try to wipe out recent transactions up to the point when next one would not contain added device. You can try setting vfs.zfs.recover to 1, however that never fixed anything for me. > If I can't tell the ZFS to ignore or delete the missing vdev, perhaps I can replace the missing with a "new" one? I have the three 1.5TB discs I wanted to add to the pool. So I can create a raidz-vdev out of them and replace the missing-vdev with the new-raidz-vdev... Can someone tell me how I am supposed to do this via CLI? > > Another idea is this: If I delete the pool, is there a way to take the three 3TB discs with my data and create a "new-old" vdev/pool out of them? I think of this because of the constructed case I mentioned... > > I still can't accept, that there's no way to rewrite or edit the metadata ;-) So am I. I started to write a readonly ZFS implementation some time ago only for recovering written data as ZFS seems to be very cautious about data and restoration is practically possible but I see no other projects for this. My code is lame and unmaintained because I have almost no spare time nowadays. You can try reading it: http://code.google.com/p/zfs-recover/ -- Sphinx of black quartz judge my vow. From owner-freebsd-fs@FreeBSD.ORG Wed Sep 12 11:20:55 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5501A1065672 for ; Wed, 12 Sep 2012 11:20:55 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) by mx1.freebsd.org (Postfix) with ESMTP id 0A0DB8FC1A for ; Wed, 12 Sep 2012 11:20:54 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1TBl04-00071p-Np for freebsd-fs@freebsd.org; Wed, 12 Sep 2012 13:20:56 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 12 Sep 2012 13:20:56 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 12 Sep 2012 13:20:56 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Ivan Voras Date: Wed, 12 Sep 2012 13:20:41 +0200 Lines: 31 Message-ID: References: <504E627C.70408@theiconic.com.au> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig5ED7E08A4B177553F06E8873" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120812 Thunderbird/14.0 In-Reply-To: <504E627C.70408@theiconic.com.au> X-Enigmail-Version: 1.4.3 Subject: Re: clustered file systems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Sep 2012 11:20:55 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig5ED7E08A4B177553F06E8873 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 10/09/2012 23:58, Jurgen Weber wrote: > This blogger mentions that their is no Clustered FS support at this > point in time for FreeBSD which I find astonishing but I have been > unable to find anything except for a GlusterFS port using fuse (not ide= al). There are several FUSE cluster file systems, but as you say, they require fuse to be actually working :( --------------enig5ED7E08A4B177553F06E8873 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAlBQcAkACgkQ/QjVBj3/HSzH9wCeMi4QgoldqLOG4h7pGnHrkyPy rZcAnjcCJQmDzJu4Z76ZBceR7wcsVnoD =/HTq -----END PGP SIGNATURE----- --------------enig5ED7E08A4B177553F06E8873-- From owner-freebsd-fs@FreeBSD.ORG Wed Sep 12 11:23:57 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3D4E8106566B; Wed, 12 Sep 2012 11:23:57 +0000 (UTC) (envelope-from nicolas@i.0x5.de) Received: from n.0x5.de (n.0x5.de [217.197.85.144]) by mx1.freebsd.org (Postfix) with ESMTP id D89028FC15; Wed, 12 Sep 2012 11:23:55 +0000 (UTC) Received: by pc5.i.0x5.de (Postfix, from userid 1003) id 3XH0xQ2n53z7ySG; Wed, 12 Sep 2012 13:23:54 +0200 (CEST) Date: Wed, 12 Sep 2012 13:23:54 +0200 From: Nicolas Rachinsky To: Andriy Gapon Message-ID: <20120912112354.GA18488@mid.pc5.i.0x5.de> References: <20120911161049.GA21635@mid.pc5.i.0x5.de> <504F6D9C.2040000@FreeBSD.org> <20120911191452.GA48029@mid.pc5.i.0x5.de> <504F967B.6030108@FreeBSD.org> <20120911201026.GA12674@mid.pc5.i.0x5.de> <50503941.3030202@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <50503941.3030202@FreeBSD.org> X-Powered-by: FreeBSD X-Homepage: http://www.rachinsky.de X-PGP-Keyid: 887BAE72 X-PGP-Fingerprint: 039E 9433 115F BC5F F88D 4524 5092 45C4 887B AE72 X-PGP-Keys: http://www.rachinsky.de/nicolas/gpg/nicolas_rachinsky.asc User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@FreeBSD.org Subject: Re: bug? with zfs, symlinks and extended attributes X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Sep 2012 11:23:57 -0000 * Andriy Gapon [2012-09-12 10:26 +0300]: > Thank you for a very good testcase. > Please try this patch. With this patch everything seems to work fine. http://svn.freebsd.org/changeset/base/240345 was applied as well. Thank you very much! Nicolas -- http://www.rachinsky.de/nicolas From owner-freebsd-fs@FreeBSD.ORG Wed Sep 12 12:11:55 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 03885106566B for ; Wed, 12 Sep 2012 12:11:55 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 520938FC08 for ; Wed, 12 Sep 2012 12:11:53 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id PAA04008; Wed, 12 Sep 2012 15:11:51 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <50507C06.6030709@FreeBSD.org> Date: Wed, 12 Sep 2012 15:11:50 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120830 Thunderbird/15.0 MIME-Version: 1.0 To: Nicolas Rachinsky References: <20120911161049.GA21635@mid.pc5.i.0x5.de> <504F6D9C.2040000@FreeBSD.org> <20120911191452.GA48029@mid.pc5.i.0x5.de> <504F967B.6030108@FreeBSD.org> <20120911201026.GA12674@mid.pc5.i.0x5.de> <50503941.3030202@FreeBSD.org> <20120912112354.GA18488@mid.pc5.i.0x5.de> In-Reply-To: <20120912112354.GA18488@mid.pc5.i.0x5.de> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org Subject: Re: bug? with zfs, symlinks and extended attributes X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Sep 2012 12:11:55 -0000 on 12/09/2012 14:23 Nicolas Rachinsky said the following: > * Andriy Gapon [2012-09-12 10:26 +0300]: >> Thank you for a very good testcase. >> Please try this patch. > > With this patch everything seems to work fine. Thank you for testing! > http://svn.freebsd.org/changeset/base/240345 was applied as well. Yes, it makes sense to have this one. > Thank you very much! It was a pleasure working with a good report :-) -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Thu Sep 13 07:25:44 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BA0D4106566B for ; Thu, 13 Sep 2012 07:25:44 +0000 (UTC) (envelope-from samu.nuutamo@gmail.com) Received: from mail-vc0-f182.google.com (mail-vc0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 6D4A78FC14 for ; Thu, 13 Sep 2012 07:25:44 +0000 (UTC) Received: by vcbfw7 with SMTP id fw7so4009658vcb.13 for ; Thu, 13 Sep 2012 00:25:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=/vJPxP6h3+D4zK+XkYcJbiyjod1QaB3oIIQVgZOUn2Y=; b=SRmVpW5mbJa8PzBLqk8MbO02oZPeLMrf3kfQ3I6Dda1qACgBOS8fiTgLsEbEbccuSi wyyEjKLeMFYdh308kjpdt8f7HgKLcd792PhB2fihWUXDCQXtDCJRDHb33b4aolb1UD+q 0IOJnkTB/5X3XQfKuthvqQSx2ILlsMK9l5ahjffLEzEj0090/RinvNgQFwCt0rZZkzCv WAiFTX0yKqEm/gDgqMUAjw3OLwuVFmrXa+aU4pJvV4ssPUQ0EC9H26rOsOyXBq7L+k6I wfCfaWGnAmh/AC3uj5Q+sCFBB3mwZMyYaIzJHnXGeLqN6DWImhPpvPstvOfyoTbAu6G0 islw== MIME-Version: 1.0 Received: by 10.220.223.13 with SMTP id ii13mr569595vcb.2.1347521143390; Thu, 13 Sep 2012 00:25:43 -0700 (PDT) Received: by 10.58.212.131 with HTTP; Thu, 13 Sep 2012 00:25:43 -0700 (PDT) Date: Thu, 13 Sep 2012 10:25:43 +0300 Message-ID: From: Samu Nuutamo To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Kernel panics with 9.0-RELEASE and fusefs-kmod 0.3.9-pre1.20080208 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Sep 2012 07:25:44 -0000 I've created a custom fuse-based filesystem that is started as a service at system startup. The filesystem itself works quite nicely but I have issues with system stability when starting up the system or restarting the service. The filesystem's usage/load mainly consists of concurrent reads. I know this is a huge post but many of these traps are "unknown" to google so I thought it would be a good idea to include every bit of information I have. The fiov_adjust trap was discussed here in freebsd-fs last december in 8.0R context but it seems to be a dead end. I'm aware of the GSoC'2011 effort to fix FUSE, but it has other issues preventing me to test it properly. For example stat must be called on the file before the contents can be read. Recently I learned about the projects/fuse branch in the SVN tree and backported the code to 9.0R, but it has the same issues as the GSoC version. This isn't a big surprise as GSoC is the base for the SVN code. Does a patch/fix/workaround exist for any of these traps (even experimental)? I'm willing to port code from the SVN and/or GSoC if someone can pinpoint the commits. -- Samu Nuutamo ------------------------------------- 1. Trap on service restart Whenever the service is restarted while some program is accessing the fs there is a high probability of a fatal trap. Fool-proof way of reproducing this is to run $ while [ 1 ]; do service myfs restart; done in first terminal $ while [ 1 ]; do find /myfs | xargs cat; done and in the second terminal. The system crashes just a few second after the find loop is started. Here's the trace: Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0xd8 fault code = supervisor write data, page not present instruction pointer = 0x20:0xffffffff81017f01 stack pointer = 0x28:0xffffff808a7295b0 frame pointer = 0x28:0xffffff808a7296c0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 1818 (find) (kgdb) bt #0 doadump (textdump=0) at /usr/src/sys/kern/kern_shutdown.c:260 #1 0xffffffff802b6e80 in db_dump (dummy=Variable "dummy" is not available.) at /usr/src/sys/ddb/db_command.c:537 #2 0xffffffff802b6501 in db_command (last_cmdp=0xffffffff8097fd60, cmd_table=Variable "cmd_table" is not available.) at /usr/src/sys/ddb/db_command.c:448 #3 0xffffffff802b6750 in db_command_loop () at /usr/src/sys/ddb/db_command.c:501 #4 0xffffffff802b8819 in db_trap (type=Variable "type" is not available.) at /usr/src/sys/ddb/db_main.c:229 #5 0xffffffff80450561 in kdb_trap (type=12, code=0, tf=0xffffff808a729500) at /usr/src/sys/kern/subr_kdb.c:620 #6 0xffffffff8063605d in trap_fatal (frame=0xffffff808a729500, eva=Variable "eva" is not available.) at /usr/src/sys/amd64/amd64/trap.c:814 #7 0xffffffff8063637e in trap_pfault (frame=0xffffff808a729500, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:735 #8 0xffffffff806367cf in trap (frame=0xffffff808a729500) at /usr/src/sys/amd64/amd64/trap.c:474 #9 0xffffffff806220af in calltrap () at /usr/src/sys/amd64/amd64/exception.S:228 #10 0xffffffff81017f01 in fuse_lookup (ap=Variable "ap" is not available.) at fuse_vnops.c:1368 #11 0xffffffff80671310 in VOP_LOOKUP_APV (vop=0xffffffff8101c320, a=0xffffff808a729720) at vnode_if.c:123 #12 0xffffffff8049c98d in lookup (ndp=0xffffff808a7299e0) at vnode_if.h:54 #13 0xffffffff8049d81c in namei (ndp=0xffffff808a7299e0) at /usr/src/sys/kern/vfs_lookup.c:297 #14 0xffffffff804b4052 in vn_open_cred (ndp=0xffffff808a7299e0, flagp=0xffffff808a7299dc, cmode=0, vn_open_flags=Variable "vn_open_flags" is not available.) at /usr/src/sys/kern/vfs_vnops.c:195 #15 0xffffffff804b12e9 in kern_openat (td=0xfffffe003162f460, fd=-100, path=0x800955abb
, pathseg=UIO_USERSPACE, flags=1, mode=Variable "mode" is not available.) at /usr/src/sys/kern/vfs_syscalls.c:1123 #16 0xffffffff8063584e in amd64_syscall (td=0xfffffe003162f460, traced=0) at subr_syscall.c:131 #17 0xffffffff80622397 in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:387 #18 0x000000080093cd2c in ?? () Previous frame inner to this frame (corrupt stack?) fuse_vnops.c:1368 points to this line on code: if (op == FUSE_GETATTR) >> cache_attrs(*vpp, (struct fuse_attr_out *)fdi.answ); ------------------------------------- 2. Trap on system startup This trap happens less frequently and most times during system startup when services are loading. By forcing a reboot loop I managed to get a trap after 1h (that's ~50 reboots). Here's the trace: Fatal trap 12: page fault while in kernel mode cpuid = 2; apic id = 02 fault virtual address = 0x40 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff81012b17 stack pointer = 0x28:0xffffff810c7ce540 frame pointer = 0x28:0xffffff810c7ce570 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 1294 (python) (kgdb) bt #0 doadump (textdump=0) at /usr/src/sys/kern/kern_shutdown.c:260 #1 0xffffffff802b6e80 in db_dump (dummy=Variable "dummy" is not available.) at /usr/src/sys/ddb/db_command.c:537 #2 0xffffffff802b6501 in db_command (last_cmdp=0xffffffff8097fd60, cmd_table=Variable "cmd_table" is not available.) at /usr/src/sys/ddb/db_command.c:448 #3 0xffffffff802b6750 in db_command_loop () at /usr/src/sys/ddb/db_command.c:501 #4 0xffffffff802b8819 in db_trap (type=Variable "type" is not available.) at /usr/src/sys/ddb/db_main.c:229 #5 0xffffffff80450561 in kdb_trap (type=12, code=0, tf=0xffffff810c7ce490) at /usr/src/sys/kern/subr_kdb.c:620 #6 0xffffffff8063605d in trap_fatal (frame=0xffffff810c7ce490, eva=Variable "eva" is not available.) at /usr/src/sys/amd64/amd64/trap.c:814 #7 0xffffffff8063637e in trap_pfault (frame=0xffffff810c7ce490, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:735 #8 0xffffffff806367cf in trap (frame=0xffffff810c7ce490) at /usr/src/sys/amd64/amd64/trap.c:474 #9 0xffffffff806220af in calltrap () at /usr/src/sys/amd64/amd64/exception.S:228 #10 0xffffffff81012b17 in fiov_adjust (fiov=0x30, size=52) at fuse_msg.c:96 #11 0xffffffff810132af in fdisp_make (fdip=0xffffff810c7ce610, mp=Variable "mp" is not available.) at fuse_msg.c:788 #12 0xffffffff81017aaa in fuse_lookup (ap=Variable "ap" is not available.) at fuse_vnops.c:1156 #13 0xffffffff80671310 in VOP_LOOKUP_APV (vop=0xffffffff8101c320, a=0xffffff810c7ce720) at vnode_if.c:123 #14 0xffffffff8049c98d in lookup (ndp=0xffffff810c7ce9e0) at vnode_if.h:54 #15 0xffffffff8049d81c in namei (ndp=0xffffff810c7ce9e0) at /usr/src/sys/kern/vfs_lookup.c:297 #16 0xffffffff804b4052 in vn_open_cred (ndp=0xffffff810c7ce9e0, flagp=0xffffff810c7ce9dc, cmode=420, vn_open_flags=Variable "vn_open_flags" is not available.) at /usr/src/sys/kern/vfs_vnops.c:195 #17 0xffffffff804b12e9 in kern_openat (td=0xfffffe000a109460, fd=-100, path=0x80075f1f2
, pathseg=UIO_USERSPACE, flags=1, mode=Variable "mode" is not available.) at /usr/src/sys/kern/vfs_syscalls.c:1123 #18 0xffffffff8063584e in amd64_syscall (td=0xfffffe000a109460, traced=0) at subr_syscall.c:131 #19 0xffffffff80622397 in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:387 #20 0x000000080073db4c in ?? () Previous frame inner to this frame (corrupt stack?) fuse_msg.c:96 points to this line on code: void fiov_adjust(struct fuse_iov *fiov, size_t size) >> { if (fiov->allocated_size < size || I have no idea what's going on there. ------------------------------------- 3. Other traps I've also seen multiple other traps with different traces, but all having a few common functions in the call stacks: fdisp_make+0xef or fdisp_make_vp+0xf4. I was about to group the fiov_adjust trap here as well as it has the same fdisp_make+0xef in the stack, but in fiov_adjust's case the fdisp_make is not the stack's topmost function call. It still looks to me that fdisp_make(_vp) is somehow broken or it uses corrupted pointers. Some call stacks (sorry, I don't have proper kernel dumps for these): (acquired from a dmesg output) Fatal trap 12: page fault while in kernel mode cpuid = 2; apic id = 02 fault virtual address = 0x40 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff81012b17 stack pointer = 0x28:0xffffff808970d680 frame pointer = 0x28:0xffffff808970d6b0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 32168 (sudo) trap number = 12 panic: page fault cpuid = 2 KDB: stack backtrace: #0 0xffffffff80460b1e at kdb_backtrace+0x5e #1 0xffffffff8042b857 at panic+0x187 #2 0xffffffff8066d6a0 at trap_fatal+0x290 #3 0xffffffff8066d9e9 at trap_pfault+0x1f9 #4 0xffffffff8066deaf at trap+0x3df #5 0xffffffff806583ef at calltrap+0x8 #6 0xffffffff81013574 at fdisp_make_vp+0xf4 #7 0xffffffff810172fd at fuse_readlink+0x4d #8 0xffffffff804b32ff at namei+0x5cf #9 0xffffffff804cb92b at vn_open_cred+0x3cb #10 0xffffffff804ca9c9 at kern_openat+0x1f9 #11 0xffffffff8066cf90 at amd64_syscall+0x450 #12 0xffffffff806586d7 at Xfast_syscall+0xf7 (manually transcribed from a jpg) Fatal trap 12: page fault while in kernel mode cpuid = 2; apic id = 00 fault virtual address = 0xc fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff8047ec95 stack pointer = 0x28:0xffffff808956a590 frame pointer = 0x28:0xffffff808956a5c0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 11418 (httpd) trap number = 12 panic: page fault cpuid = 2 KDB: stack backtrace: #0 0xffffffff80470d5e at kdb_backtrace+0x5e #1 0xffffffff8043ba97 at panic+0x187 #2 0xffffffff8067d8e0 at trap_fatal+0x290 #3 0xffffffff8067dc29 at trap_pfault+0x1f9 #4 0xffffffff8067e0ef at trap+0x3df #5 0xffffffff8066862f at calltrap+0x8 #6 0xffffffff81013574 at fdisp_make_vp+0xf4 #7 0xffffffff81018d8e at fuse_get_filehandle+0x14e #8 0xffffffff8101976e at fuse_open+0x7e #9 0xffffffff804dbaa3 at vn_open_cred+0x3cb #10 0xffffffff804dac09 at kern_openat+0x1f9 #11 0xffffffff8067d1d0 at amd64_syscall+0x450 #12 0xffffffff80668917 at Xfast_syscall+0xf7 (manually transcribed from a jpg) --- previous output not visible --- code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 6997 (cron) trap number = 12 panic: page fault cpuid = 1 KDB: stack backtrace: #0 0xffffffff80477e4e at kdb_backtrace+0x5e #1 0xffffffff80442b87 at panic+0x187 #2 0xffffffff806849d0 at trap_fatal+0x290 #3 0xffffffff80684d19 at trap_pfault+0x1f9 #4 0xffffffff806851df at trap+0x3df #5 0xffffffff8066f71f at calltrap+0x8 #6 0xffffffff81013574 at fdisp_make_vp+0xf4 #7 0xffffffff810156ea at fuse_send_release+0x6a #8 0xffffffff81015fa5 at fuse_close_f+0xf5 #9 0xffffffff80404e93 at _fdrop+0x23 #10 0xffffffff804068f2 at closef+0x52 #11 0xffffffff80407122 at kern_close+0x172 #12 0xffffffff806842c0 at amd64_syscall+0x450 #13 0xffffffff8066fa07 at Xfast_syscall+0xf7 (manually transcribed from a jpg) --- previous output not visible --- code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 1279 (sh) trap number = 12 panic: page fault cpuid = 3 KDB: stack backtrace: #0 0xffffffff80460b1e at kdb_backtrace+0x5e #1 0xffffffff8042b857 at panic+0x187 #2 0xffffffff8066d6a0 at trap_fatal+0x290 #3 0xffffffff8066d9e9 at trap_pfault+0x1f9 #4 0xffffffff8066deaf at trap+0x3df #5 0xffffffff806583ef at calltrap+0x8 #6 0xffffffff810132af at fdisp_make+0xef #7 0xffffffff81017aaa at fuse_lookup+0x44a #8 0xffffffff806a9b40 at VOP_LOOKUP_APV+0x40 #9 0xffffffff804b2154 at lookup+0x464 #10 0xffffffff804b3206 at namei+0x4d6 #11 0xffffffff804cb92b at vn_open_cred+0x3cb #10 0xffffffff804ca9c9 at kern_openat+0x1f9 #11 0xffffffff8066cf90 at amd64_syscall+0x450 #12 0xffffffff806586d7 at Xfast_syscall+0xf7 From owner-freebsd-fs@FreeBSD.ORG Fri Sep 14 05:29:38 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 46F2B1065676 for ; Fri, 14 Sep 2012 05:29:38 +0000 (UTC) (envelope-from Newsletter@goelli.de) Received: from mo6-p05-ob.rzone.de (mo6-p05-ob.rzone.de [IPv6:2a01:238:20a:202:5305::1]) by mx1.freebsd.org (Postfix) with ESMTP id A05228FC19 for ; Fri, 14 Sep 2012 05:29:37 +0000 (UTC) X-RZG-CLASS-ID: mo05 X-RZG-AUTH: :ImkTZkytb+s5KUDumTG4i0mGDH1K4fweaf9O+/5rQT5iu9PH4l7+wOwXkwRei9E= Received: from goelliNotebook (pD9FE297A.dip.t-dialin.net [217.254.41.122]) by smtp.strato.de (jored mo10) (RZmta 30.14 DYNA|AUTH) with ESMTPA id 907d7ao8E5NoYb ; Fri, 14 Sep 2012 07:29:36 +0200 (CEST) From: =?utf-8?Q?Thomas_G=C3=B6llner_=28Newsletter=29?= To: "'Volodymyr Kostyrko'" References: <001a01cd900d$bcfcc870$36f65950$@goelli.de> <504F282D.8030808@gmail.com> <000a01cd90aa$0a277310$1e765930$@goelli.de> <5050461A.9050608@gmail.com> In-Reply-To: <5050461A.9050608@gmail.com> Date: Fri, 14 Sep 2012 07:29:35 +0200 Message-ID: <000001cd9239$ed734c80$c859e580$@goelli.de> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQJeoooVG2eNN70r/NZmey5KTcQ1/AH9GaMbAUqasjACcrqh/pY5VvFg Content-Language: de Cc: freebsd-fs@freebsd.org Subject: AW: AW: ZFS: Corrupted pool metadata after adding vdev to a pool - no opportunity to rescue data from healthy vdevs? Remove a vdev? Rewrite metadata? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Sep 2012 05:29:38 -0000 Thanks for the code. It helped a little bit to understand some details. I tried the "-e" Option of "zdb" and I listed the uberblocks of the = discs with "zdb -l -u ". As I can see, the there is a "create_txg: 56482". With the uberblocks I = was able to find the corresponding date/time which is the time I added = the new vdev. But every command and option combo I'm trying gets me just two results: "Input/output error" or "Device not configured" So I can't read out data via "-bbb" or so... Perhaps there was more destroyed during the process of creating and = adding the new vdev than I thought first. It seems to me, that the = NAS4free GUI mixed up the discs during formatting or something like = that. After adding the new vdev I hat 16TB of space which was way too = much and in the info I could see the discs of the old vdev twice... But = the NAS was still playing out my data. After the restart the pool = faulted. So my last idea is the "-C" option of the "zdb" command. Perhaps I can = give a new config which it has to use. But where do I find the config = file or what is it supposed to look like? I also think there is no way to write new or edit the lables of the = discs? From owner-freebsd-fs@FreeBSD.ORG Fri Sep 14 08:35:47 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0D4421065670 for ; Fri, 14 Sep 2012 08:35:47 +0000 (UTC) (envelope-from c.kworr@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 867178FC08 for ; Fri, 14 Sep 2012 08:35:46 +0000 (UTC) Received: by bkcje9 with SMTP id je9so1009634bkc.13 for ; Fri, 14 Sep 2012 01:35:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=SjBeU2AcLnrZhfUuW4GZeg18oCq6NiqzGlqOzcdFKXw=; b=sDLTSx34x/Mse8A75iQCdtWbmgGrvnZX6bn1V5HdRlxDqZOI13/3xKLzZX+4+K9jHG gP7dbSLVWR32f8IEFQt9ckbpL8AYaXGd8TFGREjA2DoThlm/3S47vFHF79/op/cmUjLr yQDGQzhY0rC05QzqbVY36McjgxXAEMI/r0FpwT1YHNu5tKy1cAR7nIgg03uydjaLiilS rnmXqIj0euGg77MQX/Kh+/JtQExWlNHqr+wpCO15cReuJLzlsET7IjSa9S64S4kbMi61 v/mUhUUt/EOh1xQb/zrl5hpr+3MEYZzWWuFx43hG2w10PUqTW7c88W3I6Bkwpf4uWKoi cceA== Received: by 10.204.154.214 with SMTP id p22mr664669bkw.111.1347611745313; Fri, 14 Sep 2012 01:35:45 -0700 (PDT) Received: from green.local (155-10-132-95.pool.ukrtel.net. [95.132.10.155]) by mx.google.com with ESMTPS id n5sm366704bkv.14.2012.09.14.01.35.43 (version=SSLv3 cipher=OTHER); Fri, 14 Sep 2012 01:35:44 -0700 (PDT) Message-ID: <5052EC5D.4060403@gmail.com> Date: Fri, 14 Sep 2012 11:35:41 +0300 From: Volodymyr Kostyrko User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120911 Thunderbird/15.0.1 MIME-Version: 1.0 To: =?UTF-8?B?IlRob21hcyBHw7ZsbG5lciAoTmV3c2xldHRlciki?= References: <001a01cd900d$bcfcc870$36f65950$@goelli.de> <504F282D.8030808@gmail.com> <000a01cd90aa$0a277310$1e765930$@goelli.de> <5050461A.9050608@gmail.com> <000001cd9239$ed734c80$c859e580$@goelli.de> In-Reply-To: <000001cd9239$ed734c80$c859e580$@goelli.de> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-fs@freebsd.org Subject: Re: AW: AW: ZFS: Corrupted pool metadata after adding vdev to a pool - no opportunity to rescue data from healthy vdevs? Remove a vdev? Rewrite metadata? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Sep 2012 08:35:47 -0000 14.09.2012 08:29, Thomas Göllner (Newsletter) wrote: > Thanks for the code. It helped a little bit to understand some details. > > I tried the "-e" Option of "zdb" and I listed the uberblocks of the discs with "zdb -l -u ". > As I can see, the there is a "create_txg: 56482". With the uberblocks I was able to find the corresponding date/time which is the time I added the new vdev. > > But every command and option combo I'm trying gets me just two results: > "Input/output error" or > "Device not configured" > So I can't read out data via "-bbb" or so... > > Perhaps there was more destroyed during the process of creating and adding the new vdev than I thought first. It seems to me, that the NAS4free GUI mixed up the discs during formatting or something like that. After adding the new vdev I hat 16TB of space which was way too much and in the info I could see the discs of the old vdev twice... But the NAS was still playing out my data. After the restart the pool faulted. > > So my last idea is the "-C" option of the "zdb" command. Perhaps I can give a new config which it has to use. But where do I find the config file or what is it supposed to look like? Try reading Max Bruning's blog at http://mbruning.blogspot.com/, there's a good data walk on raidz and raidz on-disk format. > I also think there is no way to write new or edit the lables of the discs? This idea is called Block Pointer Rewrite and is not implemented yet. I have found no code to do that. -- Sphinx of black quartz judge my vow. From owner-freebsd-fs@FreeBSD.ORG Fri Sep 14 12:25:39 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE9ED1065673 for ; Fri, 14 Sep 2012 12:25:38 +0000 (UTC) (envelope-from Newsletter@goelli.de) Received: from mo6-p05-ob.rzone.de (mo6-p05-ob.rzone.de [IPv6:2a01:238:20a:202:5305::1]) by mx1.freebsd.org (Postfix) with ESMTP id 51E7C8FC15 for ; Fri, 14 Sep 2012 12:25:38 +0000 (UTC) X-RZG-CLASS-ID: mo05 X-RZG-AUTH: :ImkTZkytb+s5KUDumTG4i0mGDH1K4fweaf9O+/5rQT5iu9PH4l7+wOwXkwRei9E= Received: from goelliNotebook (pD9FE297A.dip.t-dialin.net [217.254.41.122]) by smtp.strato.de (jorabe mo29) (RZmta 30.14 DYNA|AUTH) with ESMTPA id u0708do8EBFeyo ; Fri, 14 Sep 2012 14:25:36 +0200 (CEST) From: =?utf-8?Q?Thomas_G=C3=B6llner_=28Newsletter=29?= To: "'Volodymyr Kostyrko'" References: <001a01cd900d$bcfcc870$36f65950$@goelli.de> <504F282D.8030808@gmail.com> <000a01cd90aa$0a277310$1e765930$@goelli.de> <5050461A.9050608@gmail.com> <000001cd9239$ed734c80$c859e580$@goelli.de> <5052EC5D.4060403@gmail.com> In-Reply-To: <5052EC5D.4060403@gmail.com> Date: Fri, 14 Sep 2012 14:25:35 +0200 Message-ID: <000a01cd9274$0aa0bba0$1fe232e0$@goelli.de> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQJeoooVG2eNN70r/NZmey5KTcQ1/AH9GaMbAUqasjACcrqh/gFBByGfA1g04pWWFQP3UA== Content-Language: de Cc: freebsd-fs@freebsd.org Subject: AW: AW: AW: ZFS: Corrupted pool metadata after adding vdev to a pool - no opportunity to rescue data from healthy vdevs? Remove a vdev? Rewrite metadata? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Sep 2012 12:25:39 -0000 > Try reading Max Bruning's blog at http://mbruning.blogspot.com/, = there's a good data walk on raidz and raidz on-disk format. I'll do that, thanks for pointing me to it. >> I also think there is no way to write new or edit the lables of the = discs? >This idea is called Block Pointer Rewrite and is not implemented yet. I = have found no code to do that. I thought it may come to this -.- Because during my last reading I had = to learn, that I have to find the "root block pointer" to recover the = maybe overwritten labels... As it changes place and content with each = copy on write process (each txg?) it will be a search for the needle in = the haystack... From owner-freebsd-fs@FreeBSD.ORG Fri Sep 14 12:27:59 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9CBA2106566C for ; Fri, 14 Sep 2012 12:27:59 +0000 (UTC) (envelope-from c.kworr@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 2065A8FC16 for ; Fri, 14 Sep 2012 12:27:58 +0000 (UTC) Received: by bkcje9 with SMTP id je9so1137157bkc.13 for ; Fri, 14 Sep 2012 05:27:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=KW9WICvYC+wXUMCgMu1rm6NKMVe3m8bvNzpCQaX1IXo=; b=xaGqFdDUfObjJuPwUVbmf7h+dLExf6bBvOzICZVKP6kyLj6d6/JDGYTEszpJVcsCVA gTHx9KNHURR4W5xwDfMIsZ4bcklNbDx9Rmbk/s5bxWGX5zmjGMMyagYSTcteZLXHEV64 9qPBJNnc39tJEmybTTIDUnXGHWJF2QwCwGpYqtgUQP6Hd9739/Dy3qjpR2Uv2TKO/x5u NDQrzpgBU6EvC3p4OMKvMBUry4p4XsBHFRq//Vgdz7pDemlQSfVDyxJGn6nxMGEdAo7q /IB8ab+HAE/o+kKLzRB5KzJYU331yOADqVQ899omhwtIoqJ2KsIieWZSjqUMLAseVbpp Zxew== Received: by 10.204.129.14 with SMTP id m14mr1101396bks.7.1347625677503; Fri, 14 Sep 2012 05:27:57 -0700 (PDT) Received: from green.local (155-10-132-95.pool.ukrtel.net. [95.132.10.155]) by mx.google.com with ESMTPS id y20sm991281bkv.11.2012.09.14.05.27.55 (version=SSLv3 cipher=OTHER); Fri, 14 Sep 2012 05:27:56 -0700 (PDT) Message-ID: <505322C9.70200@gmail.com> Date: Fri, 14 Sep 2012 15:27:53 +0300 From: Volodymyr Kostyrko User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120911 Thunderbird/15.0.1 MIME-Version: 1.0 To: =?UTF-8?B?IlRob21hcyBHw7ZsbG5lciAoTmV3c2xldHRlciki?= References: <001a01cd900d$bcfcc870$36f65950$@goelli.de> <504F282D.8030808@gmail.com> <000a01cd90aa$0a277310$1e765930$@goelli.de> <5050461A.9050608@gmail.com> <000001cd9239$ed734c80$c859e580$@goelli.de> <5052EC5D.4060403@gmail.com> <000a01cd9274$0aa0bba0$1fe232e0$@goelli.de> In-Reply-To: <000a01cd9274$0aa0bba0$1fe232e0$@goelli.de> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-fs@freebsd.org Subject: Re: AW: AW: AW: ZFS: Corrupted pool metadata after adding vdev to a pool - no opportunity to rescue data from healthy vdevs? Remove a vdev? Rewrite metadata? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Sep 2012 12:27:59 -0000 14.09.2012 15:25, Thomas Göllner (Newsletter) wrote: >> Try reading Max Bruning's blog at http://mbruning.blogspot.com/, there's a good data walk on raidz and raidz on-disk format. > > I'll do that, thanks for pointing me to it. > >>> I also think there is no way to write new or edit the lables of the discs? > > >> This idea is called Block Pointer Rewrite and is not implemented yet. I have found no code to do that. > > I thought it may come to this -.- Because during my last reading I had to learn, that I have to find the "root block pointer" to recover the maybe overwritten labels... As it changes place and content with each copy on write process (each txg?) it will be a search for the needle in the haystack... Not at all, what are you referring to is MOS and the one is contained in each UberBlock. -- Sphinx of black quartz judge my vow. From owner-freebsd-fs@FreeBSD.ORG Sat Sep 15 19:25:53 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BF2B9106566C for ; Sat, 15 Sep 2012 19:25:53 +0000 (UTC) (envelope-from Newsletter@goelli.de) Received: from mo6-p05-ob.rzone.de (mo6-p05-ob.rzone.de [IPv6:2a01:238:20a:202:5305::1]) by mx1.freebsd.org (Postfix) with ESMTP id 1D3A48FC0A for ; Sat, 15 Sep 2012 19:25:52 +0000 (UTC) X-RZG-CLASS-ID: mo05 X-RZG-AUTH: :ImkTZkytb+s5KUDumTG4i0mGDH1K4fweaf9O+/5rQT5ns8rb41Pk1w+AIjuAhQ== Received: from goelliNotebook (p4FEE5DCC.dip.t-dialin.net [79.238.93.204]) by smtp.strato.de (josoe mo22) (RZmta 30.14 DYNA|AUTH) with ESMTPA id R00920o8FJ6TP0 ; Sat, 15 Sep 2012 21:25:51 +0200 (CEST) From: =?utf-8?Q?Thomas_G=C3=B6llner_=28Newsletter=29?= To: "'Volodymyr Kostyrko'" References: <001a01cd900d$bcfcc870$36f65950$@goelli.de> <504F282D.8030808@gmail.com> <000a01cd90aa$0a277310$1e765930$@goelli.de> <5050461A.9050608@gmail.com> <000001cd9239$ed734c80$c859e580$@goelli.de> <5052EC5D.4060403@gmail.com> <000a01cd9274$0aa0bba0$1fe232e0$@goelli.de> <505322C9.70200@gmail.com> In-Reply-To: <505322C9.70200@gmail.com> Date: Sat, 15 Sep 2012 21:25:49 +0200 Message-ID: <000001cd9377$e9e9b010$bdbd1030$@goelli.de> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQJeoooVG2eNN70r/NZmey5KTcQ1/AH9GaMbAUqasjACcrqh/gFBByGfA1g04pUCQLd3HAGGeVDSlfjO6CA= Content-Language: de Cc: freebsd-fs@freebsd.org Subject: AW: AW: AW: AW: ZFS: Corrupted pool metadata after adding vdev to a pool - no opportunity to rescue data from healthy vdevs? Remove a vdev? Rewrite metadata? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Sep 2012 19:25:53 -0000 >>> Try reading Max Bruning's blog at http://mbruning.blogspot.com/, = there's a good data walk on raidz and raidz on-disk format. >> >> I'll do that, thanks for pointing me to it. Ok... I did read some of it. And I understand so much that I see, that = what I want is possible, but far beyond my skills. >>>> I also think there is no way to write new or edit the lables of the = discs? >>> >>> This idea is called Block Pointer Rewrite and is not implemented = yet. I have found no code to do that. >> >> I thought it may come to this -.- Because during my last reading I = had to learn, that I have to find the "root block pointer" to recover = the maybe overwritten labels... As it changes place and content with = each copy on write process (each txg?) it will be a search for the = needle in the haystack... > > Not at all, what are you referring to is MOS and the one is contained = in each UberBlock. So as this thing is so far beyond my skills, I am sad to point out that = I have to give up here. Without someone who will take me by the hand and = say what to do step by step I think recovering/rewriting the right = labels of my discs is something I will not be able to do within one year = or so. It's a pity that ZFS still has no tools for recovering metadata = built in. This would be a task to think of in future. Thanks again for your help Volodymyr. It is a bit of consolation that at = least I know now that I have done everything I could.