Extract Folder names from hadoop

Ebates Coupons and Cash Back


If you are looking to extract all the folder names under a certain directory in hadoop, here is the command:

Say you are looking under Directory /user/{username}, the ‘hadoop fs -ls /user/{username} gives the below output’ .

drwxr-xr-x   - user  user  0 2017-08-30 13:44 /user/{username}/product
drwxr-xr-x   - user  user  0 2017-09-01 13:39 /user/{username}/quote_lines

And the below command:

hadoop fs -ls /user/{username} | grep "^d" | awk -F':[0-9]* ' '/:/{print $2}' | cut -d'/' -f5

gives the directories:

product
quote_lines

Ebates Coupons and Cash Back