Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hadoop root equal to access root issue with directory path #96

Open
brittup opened this issue Sep 30, 2021 · 1 comment · May be fixed by #115
Open

hadoop root equal to access root issue with directory path #96

brittup opened this issue Sep 30, 2021 · 1 comment · May be fixed by #115
Assignees

Comments

@brittup
Copy link
Collaborator

brittup commented Sep 30, 2021

isi hdfs settings view --zone=az6
Service: Yes
Default Block Size: 128M
Default Checksum Type: none
Authentication Mode: simple_only
Root Directory: /ifs/az6
WebHDFS Enabled: Yes
Ambari Server:
Ambari Namenode:
ODP Version:
Data Transfer Cipher: none
Ambari Metrics Collector:

[INFO] Had this been for real, this is what would have happened...
[WARNING] The HDFS root is the same as the zone root.
[INFO] mkdir '/ifs/az6'
[INFO] chmod '755' '/ifs/az6'
[INFO] chown 'hdfs:hadoop' '/ifs/az6'
[INFO] mkdir '/ifs/az6hbase' < -- missing '/' between the root and directory
[INFO] chmod '755' '/ifs/az6hbase'
[INFO] chown 'hbase:hbase' '/ifs/az6hbase'
[INFO] mkdir '/ifs/az6ranger'

isi hdfs settings view --zone=az6
Service: Yes
Default Block Size: 128M
Default Checksum Type: none
Authentication Mode: simple_only
Root Directory: /ifs/az6/hadoop_root
WebHDFS Enabled: Yes
Ambari Server:
Ambari Namenode:
ODP Version:
Data Transfer Cipher: none
Ambari Metrics Collector:

not an issue when hadoop root is not the access zone root

[INFO] Had this been for real, this is what would have happened...
[INFO] mkdir '/ifs/az6/hadoop_root/'
[INFO] chmod '755' '/ifs/az6/hadoop_root/'
[INFO] chown 'hdfs:hadoop' '/ifs/az6/hadoop_root/'
[INFO] mkdir '/ifs/az6/hadoop_root/hbase'
[INFO] chmod '755' '/ifs/az6/hadoop_root/hbase'
[INFO] chown 'hbase:hbase' '/ifs/az6/hadoop_root/hbase'
[INFO] mkdir '/ifs/az6/hadoop_root/ranger'

@tucked
Copy link
Member

tucked commented Oct 1, 2021

I suspect this is cosmetic...

zone_root = hdfs_root = "/ifs/az6"

zone_hdfs = hdfs_root[len(zone_root):]

  • zone_hdfs = ""

path = posixpath.join(zone_hdfs, directory.path.lstrip(posixpath.sep))

  • path = "hbase"

LOGGER.info("mkdir '%s%s'", zone_root, path)

[INFO] mkdir '/ifs/az6hbase'

  • 🐛 There's no join!

(mkdir or self.onefs.mkdir)(path, directory.mode, zone=self.onefs_zone)

def mkdir(self, path, mode, recursive=False, overwrite=False, zone=None):
"""Create a directory at a (zone-root-relative) path with the given (integer) mode."""
real_path = self._zone_real_path(path, zone=zone or self.default_zone)

def _zone_real_path(self, path, zone=None):
return posixpath.join(
self.zone_settings(zone=zone or self.default_zone)['path'],
path.lstrip(posixpath.sep),
)

  • "/ifs/az6/hbase"

@tucked tucked self-assigned this Oct 1, 2021
@tucked tucked linked a pull request Aug 7, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants