Powered By Blogger

Sunday, September 29, 2019

Script to copy file in hdfs

nohup sh -x movefiles-15001-20000.sh > movefiles-15001-20000.txt &


movefiles-15001-20000.sh
Below script has code for reading the file line by line and copyinh



#!/bin/bash
cat /home_dir/basan/archive/15001-20000.txt | while read line
do
hdfs dfs -cp $line /common/basan/landing/Commitment_Loss/archive_15001_20000
if [ $? -ne 0 ]; then
                  echo -e "failure" >> /home_dir/basan/archive/failure15001-20000.txt
          else
                  echo -e "success $line" >> /home_dir/basan/archive/successfile15001-20000.txt
     fi
done


15001-20000.txt
/common/basan/landing/archive/demand/1-1-1565008054737.txt
/common/basan/landing/archive/demand/1-1-1565008054869.txt
/common/basan/landing/archive/demand/1-1-1565008054938.txt
/common/basan/landing/archive/demand/1-1-1565008055010.txt
/common/basan/landing/archive/demand/1-1-1565008055087.txt
/common/basan/landing/archive/demand/1-1-1565009497848.txt

No comments:

Post a Comment