无法写入Hadoop DFS目录模式775组权限UserGroupInformation

问题描述

我在启用了文件系统权限的专用群集上运行Hadoop 2.6.2。群集只有系统用户的密码文件，如hadoop，没有个人帐户。我正在从拥有像我的个人帐户（'clott'）的linux边缘节点访问DFS。

问题在于我无法写入模式为775和组hadoop的DFS目录（'shared'）;边缘节点显示我是hadoop组的成员。我认为边缘节点上的用户组成员会被使用，但不是吗？这是我看到的：

pre $ code clott @ edge $ id
uid = 1003（clott）gid = 1003（clott） group = 1003（clott），27（sudo），1001（hadoop）

clott @ edge $ hdfs dfs -ls / user / hadoop
找到3项
drwxr-xr -x - hadoop hadoop 0 2015-12-03 16:55 / user / hadoop / DistributedShell
drwxrwxr-x - hadoop hadoop 0 2015-12-22 09:25 / user / hadoop / shared

clott @ edge $ hdfs dfs -mkdir / user / hadoop / shared / foo
mkdir：权限被拒绝：user = clott，access = WRITE，inode =/ user / hadoop / shared：hadoop：hadoop ：drwxrwxr-x

我检查了名称节点并在其日志中发现了此警告：

  定义 hadoop.security.group.mapping （如）是：
为了实现您的目标，您需要在NameNode计算机上添加您的用户帐户（clott），并将其添加到hadoop group。
 
 
 
如果您要与您的用户一起运行MapReduce，您还需要在NodeManager主机上配置您的用户帐户。
 
I'm running Hadoop 2.6.2 on a private cluster with file-system permissions enabled. The cluster has password files with only system users like hadoop, no personal accounts. I'm accessing DFS from a linux edge node that has personal accounts like mine ('clott').  

The problem is that I cannot write to a DFS directory ('shared') that is mode 775 and group hadoop; the edge node shows me a member of hadoop group.  I thought the user-group memberships on the edge node would be used, but no?  Here's what I see:
clott@edge$ id
uid=1003(clott) gid=1003(clott) groups=1003(clott),27(sudo),1001(hadoop)

clott@edge$ hdfs dfs -ls /user/hadoop
Found 3 items
drwxr-xr-x   - hadoop hadoop          0 2015-12-03 16:55 /user/hadoop/DistributedShell
drwxrwxr-x   - hadoop hadoop          0 2015-12-22 09:25 /user/hadoop/shared

clott@edge$ hdfs dfs -mkdir /user/hadoop/shared/foo
mkdir: Permission denied: user=clott, access=WRITE, inode="/user/hadoop/shared":hadoop:hadoop:drwxrwxr-x
I checked the name node and found this warning in its log:
2015-12-22 09:32:40,941 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user clott
That makes some sense - the name node doesn't grant me access because it doesn't recognize any groups for me.  
So how does the name node find group memberships? Some googling seems to suggest I have to add my user name and various groups to the cluster but that didn't sound right. Of course this works if I set the DFS directory mode to 777, but that's not right.
Please help, thanks in advance.
 解决方案 
The definition of hadoop.security.group.mapping (as written here) is:
To accomplish your goal you'd need to add your user account (clott) on the NameNode machine and add it to hadoop group there.
If you are going to run MapReduce with your user, you'd need your user account to be configured on NodeManager hosts as well.
                        
这篇关于无法写入Hadoop DFS目录模式775组权限UserGroupInformation的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！