Copying files from local to HDFS

We can copy files from local file system to HDFS either by using copyFromLocal or put command.

• hdfs dfs -copyFromLocal or hdfs dfs -put – to copy files or directories from local filesystem into HDFS. We can also use hadoop fs in place of hdfs dfs.
• However, we will not be able to update or fix data in files when they are in HDFS. If we have to fix any data, we have to move file to local file system, fix data and then copy back to HDFS.
• Files will be divided into blocks and will be stored on Datanodes in distributed fashion based on block size and replication factor. We will get into the details later.

syntax is:
hdfs dfs -put <path of local file> <hdfs location>
hdfs dfs -copyFromLocal <path of local file> <hdfs location?

hdfs dfs -put /data/retail_db /user/`whoami`/retail_db
hdfs dfs -ls /user/`whoami`/retail_db

or we can do

hdfs dfs -copyFromLocal /data/retail_db/* /user/`whoami`/retail_db