top button
Flag Notify
    Connect to us
      Facebook Login
      Site Registration

Facebook Login
Site Registration

HDFS dfsclient: possible to force storage datanode ?

+2 votes
89 views

We have 52 DNs and more hundred clients they are store and read data from hdfs. One rack has 3 DNs and about 15 clients.

It is possible to force (if space is available) that these 15 clients prefer the 3 DNs located in there own rack to store and read data?

racklocation.conf with "org.apache.hadoop.net.NetworkTopology" is allready in use, but this does not help in this case

posted Aug 20, 2014 by anonymous

Looking for an answer?  Promote on:
Facebook Share Button Twitter Share Button LinkedIn Share Button

Similar Questions
0 votes

I was trying to implement a Hadoop/Spark audit tool, but l met a problem that I can't get the input file location and file name. I can get username, IP address, time, user command, all of these info from hdfs-audit.log. But When I submit a MapReduce job, I can't see input file location neither in Hadoop logs or Hadoop ResourceManager.

Does hadoop have API or log that contains these info through some configuration ?If it have, what should I configure?

0 votes

I have a basic question regarding the HDFS file read. I want to know what happens, when the following steps are followed:

  1. Client opens the file for reading and starts reading the file.
  2. In the meantime, someone deletes the file and file moves to the trash folder

Will Step 1. succeed? I feel, since the client has already opened the file and file still exists in .trash, the client should continue to read the file.

0 votes

Our hadoop cluster is using HDFS Federation, but when use the following command to report the HDFS status

$ ./hdfs dfsadmin -reportreport: FileSystem viewfs://nsX/ is not an HDFS file system
Usage: hdfs dfsadmin [-report] [-live] [-dead] [-decommissioning]

It gives me the following message that viewfs is NOT HDFS filesystem. Then how can I proceed to report the hdfs status

0 votes

The reason behind this is I want to have my custom user who can create anything on the entire hdfs file system (/).
I tried couple of links however, none of them were useful. Is there any way by adding/modifying some property tags I can do that ?

Contact Us
+91 9880187415
sales@queryhome.net
support@queryhome.net
#280, 3rd floor, 5th Main
6th Sector, HSR Layout
Bangalore-560102
Karnataka INDIA.
QUERY HOME
...