From owner-freebsd-fs@FreeBSD.ORG Thu Jul 4 19:34:15 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id AEE7538A for ; Thu, 4 Jul 2013 19:34:15 +0000 (UTC) (envelope-from osharoiko@gmail.com) Received: from mail-ve0-x234.google.com (mail-ve0-x234.google.com [IPv6:2607:f8b0:400c:c01::234]) by mx1.freebsd.org (Postfix) with ESMTP id 72F041B3A for ; Thu, 4 Jul 2013 19:34:15 +0000 (UTC) Received: by mail-ve0-f180.google.com with SMTP id pa12so1244096veb.39 for ; Thu, 04 Jul 2013 12:34:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=6rIrPpzPmfN2r/GUUWGwmIA5MntbXonCW2dIqzbq2go=; b=feMZd97Vd2V0FJQWfsK7sTY7aIzkA1Xb1bnIc8+JMUJ9YpvnSadGlHbhe3rwJ5CVs7 PQ4vJivMKZAxQcv4onIaMUwZcFMffmXpqxkLOABCFjMvtEvzB3wDey0QQeyxKaLN4bHL qramrKTp+M5SrGxWw7ZSHwqWdqacIna/5HxkXpFwqpPlQbjbo7vz6Zlb3BnOpCzQFujx fpUxPOV1K7QgIDT93dTB+cB7WR2/h0XQRWOxQ42y6yjzuXzuZaxBgrnCgkv3q476nD5w 7Qtc0gq4kEo8Xu6jrokplyvw7utI9Uzko3bg5eo44qmfRgBs7WoqZf+jWeMXPWo8/i05 gf9g== MIME-Version: 1.0 X-Received: by 10.58.50.7 with SMTP id y7mr4121519ven.24.1372966454971; Thu, 04 Jul 2013 12:34:14 -0700 (PDT) Received: by 10.58.28.238 with HTTP; Thu, 4 Jul 2013 12:34:14 -0700 (PDT) Date: Thu, 4 Jul 2013 20:34:14 +0100 Message-ID: Subject: NFSv4 and Kerberos, group permission seem to be ignored From: Oleg Sharoyko To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jul 2013 19:34:15 -0000 Hello, I have a small server which runs FreeBSD 9.1 and I've is set up as NFSv4 server with kerberised NFS access. My clients are linux machines. It almost works as expected (mounting/accessing files) except for one strange issue: it looks like group permissions on files and directories are being ignored. Here's an example: Server: evendim:~ % id uid=1001(ols) gid=1001(ols) groups=1001(ols),0(wheel),60000(family) evendim:~ % ls -l /data/file1 -rw-rw---- 1 root family 6 4 Jul 18:42 /data/file1 evendim:~ % cat /data/file1 test1 evendim:~ % ls -l /data/file2 -rw------- 1 ols family 6 4 Jul 18:42 /data/file2 evendim:~ % cat /data/file2 test2 evendim:~ % ls -l /data/file3 -rw-r--r-- 1 root family 6 4 Jul 18:42 /data/file3 evendim:~ % cat /data/file3 test3 evendim:~ % cat /etc/exports V4:/ -sec=krb5 /data -sec=krb5 Client: sherlock:~ % id uid=1000(ols) gid=1000(ols) groups=1000(ols),4(adm),20(dialout),24(cdrom),25(floppy),29(audio),30(dip),44(video),46(plugdev),109(netdev),110(bluetooth),113(fuse),116(scanner),118(kismet),60000(family) sherlock:~ % sudo mount -v -t nfs4 -o sec=krb5 evendim.sharoyko.net:/data /mnt mount.nfs4: timeout set for Thu Jul 4 19:52:16 2013 mount.nfs4: trying text-based options 'sec=krb5,addr=192.168.1.3,clientaddr=192.168.1.128' sherlock:~ % ls -l /mnt/file1 -rw-rw---- 1 root family 6 Jul 4 19:42 /mnt/file1 sherlock:~ % cat /mnt/file1 cat: /mnt/file1: Permission denied sherlock:~ % ls -l /mnt/file2 -rw------- 1 ols family 6 Jul 4 19:42 /mnt/file2 sherlock:~ % cat /mnt/file2 test2 sherlock:~ % ls -l /mnt/file3 -rw-r--r-- 1 root family 6 Jul 4 19:42 /mnt/file3 sherlock:~ % cat /mnt/file3 test3 As you can see file2 is inaccessible while it has group read/write permissions, user ols belongs to group family on both client and server and user/group mapping seems to work. /data on the server is a ZFS filesystem but I've also tried UFS with the same results. I've also tried ACLs and ACLs for users do work while ACLs for groups don't seem to have any effect. Is there something that I'm doing wrong? Is this an expected behaviour? I will greatly appreciate if you can help me debugging this issue. I'll quote below captured packets that are relevant to my attempt to access file1. As you can see access is clearly denied by server but I don't understand why. No. Time Source Destination Protocol Length Info 109 5.649608 192.168.1.128 192.168.1.3 NFS 258 V4 Call (Reply In 110) LOOKUP DH:0x4dcc3776/file1 Frame 109: 258 bytes on wire (2064 bits), 258 bytes captured (2064 bits) Ethernet II, Src: GemtekTe_f6:cf:a1 (00:26:82:f6:cf:a1), Dst: Giga-Byt_db:cd:c4 (90:2b:34:db:cd:c4) Internet Protocol Version 4, Src: 192.168.1.128 (192.168.1.128), Dst: 192.168.1.3 (192.168.1.3) Transmission Control Protocol, Src Port: 726 (726), Dst Port: nfs (2049), Seq: 3337, Ack: 3193, Len: 192 Remote Procedure Call, Type:Call XID:0xba073c52 Network File System [Program Version: 4] [V4 Procedure: COMPOUND (1)] Tag: length: 0 contents: minorversion: 0 Operations (count: 4) Opcode: PUTFH (22) filehandle length: 28 [hash (CRC-32): 0x4dcc3776] decode type as: unknown filehandle: 9a7470c6deedeca50a0004000000000037d80a0000000000... Opcode: LOOKUP (15) Filename: file1 length: 5 contents: file1 fill bytes: opaque data Opcode: GETFH (10) Opcode: GETATTR (9) GETATTR4args attr_request bitmap[0] = 0x0010011a [5 attributes requested] mand_attr: FATTR4_TYPE (1) mand_attr: FATTR4_CHANGE (3) mand_attr: FATTR4_SIZE (4) mand_attr: FATTR4_FSID (8) recc_attr: FATTR4_FILEID (20) bitmap[1] = 0x0030a23a [9 attributes requested] recc_attr: FATTR4_MODE (33) recc_attr: FATTR4_NUMLINKS (35) recc_attr: FATTR4_OWNER (36) recc_attr: FATTR4_OWNER_GROUP (37) recc_attr: FATTR4_RAWDEV (41) recc_attr: FATTR4_SPACE_USED (45) recc_attr: FATTR4_TIME_ACCESS (47) recc_attr: FATTR4_TIME_METADATA (52) recc_attr: FATTR4_TIME_MODIFY (53) [Main Opcode: LOOKUP (15)] No. Time Source Destination Protocol Length Info 110 5.649870 192.168.1.3 192.168.1.128 NFS 370 V4 Reply (Call In 109) LOOKUP Frame 110: 370 bytes on wire (2960 bits), 370 bytes captured (2960 bits) Ethernet II, Src: Giga-Byt_db:cd:c4 (90:2b:34:db:cd:c4), Dst: GemtekTe_f6:cf:a1 (00:26:82:f6:cf:a1) Internet Protocol Version 4, Src: 192.168.1.3 (192.168.1.3), Dst: 192.168.1.128 (192.168.1.128) Transmission Control Protocol, Src Port: nfs (2049), Dst Port: 726 (726), Seq: 3193, Ack: 3529, Len: 304 Remote Procedure Call, Type:Reply XID:0xba073c52 Network File System [Program Version: 4] [V4 Procedure: COMPOUND (1)] Status: NFS4_OK (0) Tag: length: 0 contents: Operations (count: 4) Opcode: PUTFH (22) Status: NFS4_OK (0) Opcode: LOOKUP (15) Status: NFS4_OK (0) Opcode: GETFH (10) Status: NFS4_OK (0) Filehandle length: 28 [hash (CRC-32): 0xc0a4eeb4] decode type as: unknown filehandle: 9a7470c6deedeca50a00ed00000000001bb70d0000000000... Opcode: GETATTR (9) Status: NFS4_OK (0) GETATTR4res resok4 obj_attributes attrmask bitmap[0] = 0x0010011a [5 attributes requested] mand_attr: FATTR4_TYPE (1) mand_attr: FATTR4_CHANGE (3) mand_attr: FATTR4_SIZE (4) mand_attr: FATTR4_FSID (8) recc_attr: FATTR4_FILEID (20) bitmap[1] = 0x0030a23a [9 attributes requested] recc_attr: FATTR4_MODE (33) recc_attr: FATTR4_NUMLINKS (35) recc_attr: FATTR4_OWNER (36) recc_attr: FATTR4_OWNER_GROUP (37) recc_attr: FATTR4_RAWDEV (41) recc_attr: FATTR4_SPACE_USED (45) recc_attr: FATTR4_TIME_ACCESS (47) recc_attr: FATTR4_TIME_METADATA (52) recc_attr: FATTR4_TIME_MODIFY (53) attr_vals mand_attr: FATTR4_TYPE (1) nfs_ftype4: NF4REG (1) mand_attr: FATTR4_CHANGE (3) changeid: 96 mand_attr: FATTR4_SIZE (4) size: 6 mand_attr: FATTR4_FSID (8) fattr4_fsid fsid4.major: 3329258650 fsid4.minor: 2783768030 recc_attr: FATTR4_FILEID (20) fileid: 237 recc_attr: FATTR4_MODE (33) fattr4_mode: 0660 000. .... .... .... = Unknown .... 0... .... .... = not SUID .... .0.. .... .... = not SGID .... ..0. .... .... = not save swapped text .... ...1 .... .... = Read permission for owner .... .... 1... .... = Write permission for owner .... .... .0.. .... = no Execute permission for owner .... .... ..1. .... = Read permission for group .... .... ...1 .... = Write permission for group .... .... .... 0... = no Execute permission for group .... .... .... .0.. = no Read permission for others .... .... .... ..0. = no Write permission for others .... .... .... ...0 = no Execute permission for others recc_attr: FATTR4_NUMLINKS (35) numlinks: 1 recc_attr: FATTR4_OWNER (36) fattr4_owner: root@id.sharoyko.net length: 20 contents: root@id.sharoyko.net recc_attr: FATTR4_OWNER_GROUP (37) fattr4_owner_group: family@id.sharoyko.net length: 22 contents: family@id.sharoyko.net fill bytes: opaque data recc_attr: FATTR4_RAWDEV (41) specdata1: 128 specdata2: 123863040 recc_attr: FATTR4_SPACE_USED (45) space_used: 1024 recc_attr: FATTR4_TIME_ACCESS (47) seconds: 1372963326 nseconds: 263434280 recc_attr: FATTR4_TIME_METADATA (52) seconds: 1372963379 nseconds: 804435894 recc_attr: FATTR4_TIME_MODIFY (53) seconds: 1372963326 nseconds: 264422029 [Main Opcode: LOOKUP (15)] No. Time Source Destination Protocol Length Info 117 8.456684 192.168.1.128 192.168.1.3 NFS 322 V4 Call (Reply In 118) OPEN DH:0x4dcc3776/file1 Frame 117: 322 bytes on wire (2576 bits), 322 bytes captured (2576 bits) Ethernet II, Src: GemtekTe_f6:cf:a1 (00:26:82:f6:cf:a1), Dst: Giga-Byt_db:cd:c4 (90:2b:34:db:cd:c4) Internet Protocol Version 4, Src: 192.168.1.128 (192.168.1.128), Dst: 192.168.1.3 (192.168.1.3) Transmission Control Protocol, Src Port: 726 (726), Dst Port: nfs (2049), Seq: 3905, Ack: 3697, Len: 256 Remote Procedure Call, Type:Call XID:0xbd073c52 Network File System [Program Version: 4] [V4 Procedure: COMPOUND (1)] Tag: length: 0 contents: minorversion: 0 Operations (count: 5) Opcode: PUTFH (22) filehandle length: 28 [hash (CRC-32): 0x4dcc3776] decode type as: unknown filehandle: 9a7470c6deedeca50a0004000000000037d80a0000000000... Opcode: OPEN (18) seqid: 0x00000000 share_access: OPEN4_SHARE_ACCESS_READ (1) share_deny: OPEN4_SHARE_DENY_NONE (0) clientid: 0xcd6cc75124000000 owner: length: 24 contents: Open Type: OPEN4_NOCREATE (0) Claim Type: CLAIM_NULL (0) Filename: file1 length: 5 contents: file1 fill bytes: opaque data Opcode: GETFH (10) Opcode: ACCESS (3), [Check: RD MD XT XE] Check access: 0x2d .... ...1 = 0x01 READ: allowed? .... .1.. = 0x04 MODIFY: allowed? .... 1... = 0x08 EXTEND: allowed? ..1. .... = 0x20 EXECUTE: allowed? Opcode: GETATTR (9) GETATTR4args attr_request bitmap[0] = 0x0010011a [5 attributes requested] mand_attr: FATTR4_TYPE (1) mand_attr: FATTR4_CHANGE (3) mand_attr: FATTR4_SIZE (4) mand_attr: FATTR4_FSID (8) recc_attr: FATTR4_FILEID (20) bitmap[1] = 0x0030a23a [9 attributes requested] recc_attr: FATTR4_MODE (33) recc_attr: FATTR4_NUMLINKS (35) recc_attr: FATTR4_OWNER (36) recc_attr: FATTR4_OWNER_GROUP (37) recc_attr: FATTR4_RAWDEV (41) recc_attr: FATTR4_SPACE_USED (45) recc_attr: FATTR4_TIME_ACCESS (47) recc_attr: FATTR4_TIME_METADATA (52) recc_attr: FATTR4_TIME_MODIFY (53) [Main Opcode: OPEN (18)] No. Time Source Destination Protocol Length Info 118 8.456811 192.168.1.3 192.168.1.128 NFS 150 V4 Reply (Call In 117) OPEN Status: NFS4ERR_ACCES Frame 118: 150 bytes on wire (1200 bits), 150 bytes captured (1200 bits) Ethernet II, Src: Giga-Byt_db:cd:c4 (90:2b:34:db:cd:c4), Dst: GemtekTe_f6:cf:a1 (00:26:82:f6:cf:a1) Internet Protocol Version 4, Src: 192.168.1.3 (192.168.1.3), Dst: 192.168.1.128 (192.168.1.128) Transmission Control Protocol, Src Port: nfs (2049), Dst Port: 726 (726), Seq: 3697, Ack: 4161, Len: 84 Remote Procedure Call, Type:Reply XID:0xbd073c52 Network File System [Program Version: 4] [V4 Procedure: COMPOUND (1)] Status: NFS4ERR_ACCES (13) Tag: length: 0 contents: Operations (count: 2) Opcode: PUTFH (22) Status: NFS4_OK (0) Opcode: OPEN (18) Status: NFS4ERR_ACCES (13) [Main Opcode: OPEN (18)] Kind regards, -- Oleg