Target directory :
We will mention the name of the folder.
Target directory should not be existing
Warehouse directory :
It will create the folder with the name of
the table
[cloudera@quickstart ~]$ sqoop import --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera --table orders --target-dir /user/training/sqoop_import/retail_db/orders
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 02:11:56 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 02:11:57 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 02:11:57 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/11/02 02:11:57 INFO tool.CodeGenTool: Beginning code generation
19/11/02 02:11:58 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `orders` AS t LIMIT 1
19/11/02 02:11:59 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `orders` AS t LIMIT 1
19/11/02 02:11:59 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/3bbd10d292ff52f4bf4fd6cfc3bdebf6/orders.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/11/02 02:12:08 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/3bbd10d292ff52f4bf4fd6cfc3bdebf6/orders.jar
19/11/02 02:12:08 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/11/02 02:12:08 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/11/02 02:12:08 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/11/02 02:12:08 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/11/02 02:12:08 INFO mapreduce.ImportJobBase: Beginning import of orders
19/11/02 02:12:08 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
19/11/02 02:12:09 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/11/02 02:12:11 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/11/02 02:12:11 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/11/02 02:12:20 INFO db.DBInputFormat: Using read commited transaction isolation
19/11/02 02:12:20 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`order_id`), MAX(`order_id`) FROM `orders`
19/11/02 02:12:20 INFO db.IntegerSplitter: Split size: 17220; Num splits: 4 from: 1 to: 68883
19/11/02 02:12:20 INFO mapreduce.JobSubmitter: number of splits:4
19/11/02 02:12:21 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1572629054486_0001
19/11/02 02:12:23 INFO impl.YarnClientImpl: Submitted application application_1572629054486_0001
19/11/02 02:12:23 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1572629054486_0001/
19/11/02 02:12:23 INFO mapreduce.Job: Running job: job_1572629054486_0001
19/11/02 02:12:49 INFO mapreduce.Job: Job job_1572629054486_0001 running in uber mode : false
19/11/02 02:12:49 INFO mapreduce.Job: map 0% reduce 0%
19/11/02 02:13:34 INFO mapreduce.Job: map 25% reduce 0%
19/11/02 02:13:35 INFO mapreduce.Job: map 100% reduce 0%
19/11/02 02:13:37 INFO mapreduce.Job: Job job_1572629054486_0001 completed successfully
19/11/02 02:13:37 INFO mapreduce.Job: Counters: 30
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=685628
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=469
HDFS: Number of bytes written=2999944
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=163849
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=163849
Total vcore-milliseconds taken by all map tasks=163849
Total megabyte-milliseconds taken by all map tasks=167781376
Map-Reduce Framework
Map input records=68883
Map output records=68883
Input split bytes=469
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=1296
CPU time spent (ms)=24860
Physical memory (bytes) snapshot=986669056
Virtual memory (bytes) snapshot=6299193344
Total committed heap usage (bytes)=698351616
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=2999944
19/11/02 02:13:37 INFO mapreduce.ImportJobBase: Transferred 2.861 MB in 86.1969 seconds (33.9877 KB/sec)
19/11/02 02:13:37 INFO mapreduce.ImportJobBase: Retrieved 68883 records.
[cloudera@quickstart ~]$ sqoop import --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera --table order_items --warehouse-dir /user/training/sqoop_import/retail_db
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 02:14:40 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 02:14:40 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 02:14:41 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/11/02 02:14:41 INFO tool.CodeGenTool: Beginning code generation
19/11/02 02:14:41 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 02:14:41 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 02:14:41 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/f41e9c18fc058790f02c68162dbbc6c5/order_items.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/11/02 02:14:44 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/f41e9c18fc058790f02c68162dbbc6c5/order_items.jar
19/11/02 02:14:45 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/11/02 02:14:45 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/11/02 02:14:45 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/11/02 02:14:45 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/11/02 02:14:45 INFO mapreduce.ImportJobBase: Beginning import of order_items
19/11/02 02:14:45 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
19/11/02 02:14:45 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/11/02 02:14:46 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/11/02 02:14:47 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/11/02 02:14:54 INFO db.DBInputFormat: Using read commited transaction isolation
19/11/02 02:14:54 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`order_item_id`), MAX(`order_item_id`) FROM `order_items`
19/11/02 02:14:54 INFO db.IntegerSplitter: Split size: 43049; Num splits: 4 from: 1 to: 172198
19/11/02 02:14:54 INFO mapreduce.JobSubmitter: number of splits:4
19/11/02 02:14:55 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1572629054486_0002
19/11/02 02:14:55 INFO impl.YarnClientImpl: Submitted application application_1572629054486_0002
19/11/02 02:14:55 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1572629054486_0002/
19/11/02 02:14:55 INFO mapreduce.Job: Running job: job_1572629054486_0002
19/11/02 02:15:06 INFO mapreduce.Job: Job job_1572629054486_0002 running in uber mode : false
19/11/02 02:15:06 INFO mapreduce.Job: map 0% reduce 0%
19/11/02 02:15:30 INFO mapreduce.Job: map 25% reduce 0%
19/11/02 02:15:33 INFO mapreduce.Job: map 50% reduce 0%
19/11/02 02:15:36 INFO mapreduce.Job: map 100% reduce 0%
19/11/02 02:15:37 INFO mapreduce.Job: Job job_1572629054486_0002 completed successfully
19/11/02 02:15:37 INFO mapreduce.Job: Counters: 31
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=686028
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=512
HDFS: Number of bytes written=5408880
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Killed map tasks=1
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=98147
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=98147
Total vcore-milliseconds taken by all map tasks=98147
Total megabyte-milliseconds taken by all map tasks=100502528
Map-Reduce Framework
Map input records=172198
Map output records=172198
Input split bytes=512
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=1158
CPU time spent (ms)=25050
Physical memory (bytes) snapshot=970797056
Virtual memory (bytes) snapshot=6314131456
Total committed heap usage (bytes)=686817280
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=5408880
19/11/02 02:15:37 INFO mapreduce.ImportJobBase: Transferred 5.1583 MB in 50.8467 seconds (103.883 KB/sec)
19/11/02 02:15:37 INFO mapreduce.ImportJobBase: Retrieved 172198 records.
[cloudera@quickstart ~]$ pwd
/home/cloudera
[cloudera@quickstart ~]$ hdfs dfs -ls /user/training/sqoop_import/retail_db
Found 2 items
drwxr-xr-x - cloudera supergroup 0 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items
drwxr-xr-x - cloudera supergroup 0 2019-11-02 02:13 /user/training/sqoop_import/retail_db/orders
[cloudera@quickstart ~]$ hdfs dfs -ls /user/training/sqoop_import/retail_db/orders
Found 5 items
-rw-r--r-- 1 cloudera supergroup 0 2019-11-02 02:13 /user/training/sqoop_import/retail_db/orders/_SUCCESS
-rw-r--r-- 1 cloudera supergroup 741614 2019-11-02 02:13 /user/training/sqoop_import/retail_db/orders/part-m-00000
-rw-r--r-- 1 cloudera supergroup 753022 2019-11-02 02:13 /user/training/sqoop_import/retail_db/orders/part-m-00001
-rw-r--r-- 1 cloudera supergroup 752368 2019-11-02 02:13 /user/training/sqoop_import/retail_db/orders/part-m-00002
-rw-r--r-- 1 cloudera supergroup 752940 2019-11-02 02:13 /user/training/sqoop_import/retail_db/orders/part-m-00003
[cloudera@quickstart ~]$ hdfs dfs -ls /user/training/sqoop_import/retail_db/order_itmes
ls: `/user/training/sqoop_import/retail_db/order_itmes': No such file or directory
[cloudera@quickstart ~]$ hdfs dfs -ls /user/training/sqoop_import/retail_db/order_items
Found 5 items
-rw-r--r-- 1 cloudera supergroup 0 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/_SUCCESS
-rw-r--r-- 1 cloudera supergroup 1303818 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/part-m-00000
-rw-r--r-- 1 cloudera supergroup 1343222 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/part-m-00001
-rw-r--r-- 1 cloudera supergroup 1371917 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/part-m-00002
-rw-r--r-- 1 cloudera supergroup 1389923 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/part-m-00003
[cloudera@quickstart ~]$ sqoop import --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera --table order_items --warehouse-dir /user/training/sqoop_import/retail_db --append
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 02:18:35 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 02:18:35 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 02:18:36 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/11/02 02:18:36 INFO tool.CodeGenTool: Beginning code generation
19/11/02 02:18:36 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 02:18:36 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 02:18:36 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/de4dcc946f7582070659353794f287c6/order_items.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/11/02 02:18:39 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/de4dcc946f7582070659353794f287c6/order_items.jar
19/11/02 02:18:39 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/11/02 02:18:39 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/11/02 02:18:39 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/11/02 02:18:39 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/11/02 02:18:39 INFO mapreduce.ImportJobBase: Beginning import of order_items
19/11/02 02:18:39 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
19/11/02 02:18:40 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/11/02 02:18:41 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/11/02 02:18:41 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/11/02 02:18:46 INFO db.DBInputFormat: Using read commited transaction isolation
19/11/02 02:18:46 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`order_item_id`), MAX(`order_item_id`) FROM `order_items`
19/11/02 02:18:46 INFO db.IntegerSplitter: Split size: 43049; Num splits: 4 from: 1 to: 172198
19/11/02 02:18:47 INFO mapreduce.JobSubmitter: number of splits:4
19/11/02 02:18:47 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1572629054486_0003
19/11/02 02:18:48 INFO impl.YarnClientImpl: Submitted application application_1572629054486_0003
19/11/02 02:18:48 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1572629054486_0003/
19/11/02 02:18:48 INFO mapreduce.Job: Running job: job_1572629054486_0003
19/11/02 02:18:57 INFO mapreduce.Job: Job job_1572629054486_0003 running in uber mode : false
19/11/02 02:18:57 INFO mapreduce.Job: map 0% reduce 0%
19/11/02 02:19:26 INFO mapreduce.Job: map 50% reduce 0%
19/11/02 02:19:27 INFO mapreduce.Job: map 75% reduce 0%
19/11/02 02:19:28 INFO mapreduce.Job: map 100% reduce 0%
19/11/02 02:19:29 INFO mapreduce.Job: Job job_1572629054486_0003 completed successfully
19/11/02 02:19:30 INFO mapreduce.Job: Counters: 31
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=686092
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=512
HDFS: Number of bytes written=5408880
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Killed map tasks=1
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=104839
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=104839
Total vcore-milliseconds taken by all map tasks=104839
Total megabyte-milliseconds taken by all map tasks=107355136
Map-Reduce Framework
Map input records=172198
Map output records=172198
Input split bytes=512
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=1398
CPU time spent (ms)=26580
Physical memory (bytes) snapshot=1000693760
Virtual memory (bytes) snapshot=6280458240
Total committed heap usage (bytes)=699400192
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=5408880
19/11/02 02:19:30 INFO mapreduce.ImportJobBase: Transferred 5.1583 MB in 48.5384 seconds (108.8232 KB/sec)
19/11/02 02:19:30 INFO mapreduce.ImportJobBase: Retrieved 172198 records.
19/11/02 02:19:30 INFO util.AppendUtils: Appending to directory order_items
19/11/02 02:19:30 INFO util.AppendUtils: Using found partition 4
[cloudera@quickstart ~]$ hdfs dfs -ls /user/training/sqoop_import/retail_db/order_itemsFound 9 items
-rw-r--r-- 1 cloudera supergroup 0 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/_SUCCESS
-rw-r--r-- 1 cloudera supergroup 1303818 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/part-m-00000
-rw-r--r-- 1 cloudera supergroup 1343222 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/part-m-00001
-rw-r--r-- 1 cloudera supergroup 1371917 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/part-m-00002
-rw-r--r-- 1 cloudera supergroup 1389923 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/part-m-00003
-rw-r--r-- 1 cloudera cloudera 1303818 2019-11-02 02:19 /user/training/sqoop_import/retail_db/order_items/part-m-00004
-rw-r--r-- 1 cloudera cloudera 1343222 2019-11-02 02:19 /user/training/sqoop_import/retail_db/order_items/part-m-00005
-rw-r--r-- 1 cloudera cloudera 1371917 2019-11-02 02:19 /user/training/sqoop_import/retail_db/order_items/part-m-00006
-rw-r--r-- 1 cloudera cloudera 1389923 2019-11-02 02:19 /user/training/sqoop_import/retail_db/order_items/part-m-00007
[cloudera@quickstart ~]$ -rw-r--r-- 1 cloudera supergroup 1389923 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/part-m-00003
bash: -rw-r--r--: command not found
[cloudera@quickstart ~]$ sqoop import --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera --table order_items --warehouse-dir /user/training/sqoop_import/retail_db --delete-target-dir
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 02:21:45 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 02:21:45 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 02:21:46 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/11/02 02:21:46 INFO tool.CodeGenTool: Beginning code generation
19/11/02 02:21:46 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 02:21:47 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 02:21:47 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/6289265f19a88a163a949ccdae0e4609/order_items.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/11/02 02:21:50 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/6289265f19a88a163a949ccdae0e4609/order_items.jar
19/11/02 02:21:51 INFO tool.ImportTool: Destination directory /user/training/sqoop_import/retail_db/order_items deleted.
19/11/02 02:21:51 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/11/02 02:21:51 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/11/02 02:21:51 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/11/02 02:21:51 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/11/02 02:21:51 INFO mapreduce.ImportJobBase: Beginning import of order_items
19/11/02 02:21:51 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
19/11/02 02:21:51 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/11/02 02:21:51 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/11/02 02:21:52 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/11/02 02:21:58 INFO db.DBInputFormat: Using read commited transaction isolation
19/11/02 02:21:58 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`order_item_id`), MAX(`order_item_id`) FROM `order_items`
19/11/02 02:21:58 INFO db.IntegerSplitter: Split size: 43049; Num splits: 4 from: 1 to: 172198
19/11/02 02:21:58 INFO mapreduce.JobSubmitter: number of splits:4
19/11/02 02:21:58 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1572629054486_0004
19/11/02 02:21:59 INFO impl.YarnClientImpl: Submitted application application_1572629054486_0004
19/11/02 02:21:59 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1572629054486_0004/
19/11/02 02:21:59 INFO mapreduce.Job: Running job: job_1572629054486_0004
19/11/02 02:22:09 INFO mapreduce.Job: Job job_1572629054486_0004 running in uber mode : false
19/11/02 02:22:09 INFO mapreduce.Job: map 0% reduce 0%
19/11/02 02:22:30 INFO mapreduce.Job: map 25% reduce 0%
19/11/02 02:22:31 INFO mapreduce.Job: map 50% reduce 0%
19/11/02 02:22:34 INFO mapreduce.Job: map 100% reduce 0%
19/11/02 02:22:36 INFO mapreduce.Job: Job job_1572629054486_0004 completed successfully
19/11/02 02:22:36 INFO mapreduce.Job: Counters: 31
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=686024
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=512
HDFS: Number of bytes written=5408880
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Killed map tasks=1
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=80397
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=80397
Total vcore-milliseconds taken by all map tasks=80397
Total megabyte-milliseconds taken by all map tasks=82326528
Map-Reduce Framework
Map input records=172198
Map output records=172198
Input split bytes=512
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=658
CPU time spent (ms)=24240
Physical memory (bytes) snapshot=942854144
Virtual memory (bytes) snapshot=6264147968
Total committed heap usage (bytes)=618659840
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=5408880
19/11/02 02:22:36 INFO mapreduce.ImportJobBase: Transferred 5.1583 MB in 44.2065 seconds (119.4871 KB/sec)
19/11/02 02:22:36 INFO mapreduce.ImportJobBase: Retrieved 172198 records.
[cloudera@quickstart ~]$ hdfs dfs -ls /user/training/sqoop_import/retail_db/order_itemsFound 5 items
-rw-r--r-- 1 cloudera supergroup 0 2019-11-02 02:22 /user/training/sqoop_import/retail_db/order_items/_SUCCESS
-rw-r--r-- 1 cloudera supergroup 1303818 2019-11-02 02:22 /user/training/sqoop_import/retail_db/order_items/part-m-00000
-rw-r--r-- 1 cloudera supergroup 1343222 2019-11-02 02:22 /user/training/sqoop_import/retail_db/order_items/part-m-00001
-rw-r--r-- 1 cloudera supergroup 1371917 2019-11-02 02:22 /user/training/sqoop_import/retail_db/order_items/part-m-00002
-rw-r--r-- 1 cloudera supergroup 1389923 2019-11-02 02:22 /user/training/sqoop_import/retail_db/order_items/part-m-00003
[cloudera@quickstart ~]$
We can import data and we cam append or overwrite the content
of the table
We will mention the name of the folder.
Target directory should not be existing
Warehouse directory :
It will create the folder with the name of
the table
[cloudera@quickstart ~]$ sqoop import --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera --table orders --target-dir /user/training/sqoop_import/retail_db/orders
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 02:11:56 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 02:11:57 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 02:11:57 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/11/02 02:11:57 INFO tool.CodeGenTool: Beginning code generation
19/11/02 02:11:58 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `orders` AS t LIMIT 1
19/11/02 02:11:59 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `orders` AS t LIMIT 1
19/11/02 02:11:59 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/3bbd10d292ff52f4bf4fd6cfc3bdebf6/orders.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/11/02 02:12:08 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/3bbd10d292ff52f4bf4fd6cfc3bdebf6/orders.jar
19/11/02 02:12:08 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/11/02 02:12:08 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/11/02 02:12:08 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/11/02 02:12:08 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/11/02 02:12:08 INFO mapreduce.ImportJobBase: Beginning import of orders
19/11/02 02:12:08 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
19/11/02 02:12:09 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/11/02 02:12:11 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/11/02 02:12:11 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/11/02 02:12:20 INFO db.DBInputFormat: Using read commited transaction isolation
19/11/02 02:12:20 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`order_id`), MAX(`order_id`) FROM `orders`
19/11/02 02:12:20 INFO db.IntegerSplitter: Split size: 17220; Num splits: 4 from: 1 to: 68883
19/11/02 02:12:20 INFO mapreduce.JobSubmitter: number of splits:4
19/11/02 02:12:21 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1572629054486_0001
19/11/02 02:12:23 INFO impl.YarnClientImpl: Submitted application application_1572629054486_0001
19/11/02 02:12:23 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1572629054486_0001/
19/11/02 02:12:23 INFO mapreduce.Job: Running job: job_1572629054486_0001
19/11/02 02:12:49 INFO mapreduce.Job: Job job_1572629054486_0001 running in uber mode : false
19/11/02 02:12:49 INFO mapreduce.Job: map 0% reduce 0%
19/11/02 02:13:34 INFO mapreduce.Job: map 25% reduce 0%
19/11/02 02:13:35 INFO mapreduce.Job: map 100% reduce 0%
19/11/02 02:13:37 INFO mapreduce.Job: Job job_1572629054486_0001 completed successfully
19/11/02 02:13:37 INFO mapreduce.Job: Counters: 30
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=685628
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=469
HDFS: Number of bytes written=2999944
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=163849
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=163849
Total vcore-milliseconds taken by all map tasks=163849
Total megabyte-milliseconds taken by all map tasks=167781376
Map-Reduce Framework
Map input records=68883
Map output records=68883
Input split bytes=469
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=1296
CPU time spent (ms)=24860
Physical memory (bytes) snapshot=986669056
Virtual memory (bytes) snapshot=6299193344
Total committed heap usage (bytes)=698351616
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=2999944
19/11/02 02:13:37 INFO mapreduce.ImportJobBase: Transferred 2.861 MB in 86.1969 seconds (33.9877 KB/sec)
19/11/02 02:13:37 INFO mapreduce.ImportJobBase: Retrieved 68883 records.
[cloudera@quickstart ~]$ sqoop import --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera --table order_items --warehouse-dir /user/training/sqoop_import/retail_db
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 02:14:40 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 02:14:40 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 02:14:41 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/11/02 02:14:41 INFO tool.CodeGenTool: Beginning code generation
19/11/02 02:14:41 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 02:14:41 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 02:14:41 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/f41e9c18fc058790f02c68162dbbc6c5/order_items.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/11/02 02:14:44 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/f41e9c18fc058790f02c68162dbbc6c5/order_items.jar
19/11/02 02:14:45 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/11/02 02:14:45 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/11/02 02:14:45 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/11/02 02:14:45 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/11/02 02:14:45 INFO mapreduce.ImportJobBase: Beginning import of order_items
19/11/02 02:14:45 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
19/11/02 02:14:45 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/11/02 02:14:46 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/11/02 02:14:47 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/11/02 02:14:54 INFO db.DBInputFormat: Using read commited transaction isolation
19/11/02 02:14:54 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`order_item_id`), MAX(`order_item_id`) FROM `order_items`
19/11/02 02:14:54 INFO db.IntegerSplitter: Split size: 43049; Num splits: 4 from: 1 to: 172198
19/11/02 02:14:54 INFO mapreduce.JobSubmitter: number of splits:4
19/11/02 02:14:55 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1572629054486_0002
19/11/02 02:14:55 INFO impl.YarnClientImpl: Submitted application application_1572629054486_0002
19/11/02 02:14:55 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1572629054486_0002/
19/11/02 02:14:55 INFO mapreduce.Job: Running job: job_1572629054486_0002
19/11/02 02:15:06 INFO mapreduce.Job: Job job_1572629054486_0002 running in uber mode : false
19/11/02 02:15:06 INFO mapreduce.Job: map 0% reduce 0%
19/11/02 02:15:30 INFO mapreduce.Job: map 25% reduce 0%
19/11/02 02:15:33 INFO mapreduce.Job: map 50% reduce 0%
19/11/02 02:15:36 INFO mapreduce.Job: map 100% reduce 0%
19/11/02 02:15:37 INFO mapreduce.Job: Job job_1572629054486_0002 completed successfully
19/11/02 02:15:37 INFO mapreduce.Job: Counters: 31
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=686028
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=512
HDFS: Number of bytes written=5408880
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Killed map tasks=1
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=98147
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=98147
Total vcore-milliseconds taken by all map tasks=98147
Total megabyte-milliseconds taken by all map tasks=100502528
Map-Reduce Framework
Map input records=172198
Map output records=172198
Input split bytes=512
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=1158
CPU time spent (ms)=25050
Physical memory (bytes) snapshot=970797056
Virtual memory (bytes) snapshot=6314131456
Total committed heap usage (bytes)=686817280
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=5408880
19/11/02 02:15:37 INFO mapreduce.ImportJobBase: Transferred 5.1583 MB in 50.8467 seconds (103.883 KB/sec)
19/11/02 02:15:37 INFO mapreduce.ImportJobBase: Retrieved 172198 records.
[cloudera@quickstart ~]$ pwd
/home/cloudera
[cloudera@quickstart ~]$ hdfs dfs -ls /user/training/sqoop_import/retail_db
Found 2 items
drwxr-xr-x - cloudera supergroup 0 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items
drwxr-xr-x - cloudera supergroup 0 2019-11-02 02:13 /user/training/sqoop_import/retail_db/orders
[cloudera@quickstart ~]$ hdfs dfs -ls /user/training/sqoop_import/retail_db/orders
Found 5 items
-rw-r--r-- 1 cloudera supergroup 0 2019-11-02 02:13 /user/training/sqoop_import/retail_db/orders/_SUCCESS
-rw-r--r-- 1 cloudera supergroup 741614 2019-11-02 02:13 /user/training/sqoop_import/retail_db/orders/part-m-00000
-rw-r--r-- 1 cloudera supergroup 753022 2019-11-02 02:13 /user/training/sqoop_import/retail_db/orders/part-m-00001
-rw-r--r-- 1 cloudera supergroup 752368 2019-11-02 02:13 /user/training/sqoop_import/retail_db/orders/part-m-00002
-rw-r--r-- 1 cloudera supergroup 752940 2019-11-02 02:13 /user/training/sqoop_import/retail_db/orders/part-m-00003
[cloudera@quickstart ~]$ hdfs dfs -ls /user/training/sqoop_import/retail_db/order_itmes
ls: `/user/training/sqoop_import/retail_db/order_itmes': No such file or directory
[cloudera@quickstart ~]$ hdfs dfs -ls /user/training/sqoop_import/retail_db/order_items
Found 5 items
-rw-r--r-- 1 cloudera supergroup 0 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/_SUCCESS
-rw-r--r-- 1 cloudera supergroup 1303818 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/part-m-00000
-rw-r--r-- 1 cloudera supergroup 1343222 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/part-m-00001
-rw-r--r-- 1 cloudera supergroup 1371917 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/part-m-00002
-rw-r--r-- 1 cloudera supergroup 1389923 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/part-m-00003
[cloudera@quickstart ~]$ sqoop import --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera --table order_items --warehouse-dir /user/training/sqoop_import/retail_db --append
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 02:18:35 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 02:18:35 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 02:18:36 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/11/02 02:18:36 INFO tool.CodeGenTool: Beginning code generation
19/11/02 02:18:36 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 02:18:36 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 02:18:36 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/de4dcc946f7582070659353794f287c6/order_items.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/11/02 02:18:39 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/de4dcc946f7582070659353794f287c6/order_items.jar
19/11/02 02:18:39 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/11/02 02:18:39 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/11/02 02:18:39 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/11/02 02:18:39 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/11/02 02:18:39 INFO mapreduce.ImportJobBase: Beginning import of order_items
19/11/02 02:18:39 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
19/11/02 02:18:40 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/11/02 02:18:41 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/11/02 02:18:41 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/11/02 02:18:46 INFO db.DBInputFormat: Using read commited transaction isolation
19/11/02 02:18:46 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`order_item_id`), MAX(`order_item_id`) FROM `order_items`
19/11/02 02:18:46 INFO db.IntegerSplitter: Split size: 43049; Num splits: 4 from: 1 to: 172198
19/11/02 02:18:47 INFO mapreduce.JobSubmitter: number of splits:4
19/11/02 02:18:47 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1572629054486_0003
19/11/02 02:18:48 INFO impl.YarnClientImpl: Submitted application application_1572629054486_0003
19/11/02 02:18:48 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1572629054486_0003/
19/11/02 02:18:48 INFO mapreduce.Job: Running job: job_1572629054486_0003
19/11/02 02:18:57 INFO mapreduce.Job: Job job_1572629054486_0003 running in uber mode : false
19/11/02 02:18:57 INFO mapreduce.Job: map 0% reduce 0%
19/11/02 02:19:26 INFO mapreduce.Job: map 50% reduce 0%
19/11/02 02:19:27 INFO mapreduce.Job: map 75% reduce 0%
19/11/02 02:19:28 INFO mapreduce.Job: map 100% reduce 0%
19/11/02 02:19:29 INFO mapreduce.Job: Job job_1572629054486_0003 completed successfully
19/11/02 02:19:30 INFO mapreduce.Job: Counters: 31
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=686092
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=512
HDFS: Number of bytes written=5408880
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Killed map tasks=1
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=104839
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=104839
Total vcore-milliseconds taken by all map tasks=104839
Total megabyte-milliseconds taken by all map tasks=107355136
Map-Reduce Framework
Map input records=172198
Map output records=172198
Input split bytes=512
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=1398
CPU time spent (ms)=26580
Physical memory (bytes) snapshot=1000693760
Virtual memory (bytes) snapshot=6280458240
Total committed heap usage (bytes)=699400192
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=5408880
19/11/02 02:19:30 INFO mapreduce.ImportJobBase: Transferred 5.1583 MB in 48.5384 seconds (108.8232 KB/sec)
19/11/02 02:19:30 INFO mapreduce.ImportJobBase: Retrieved 172198 records.
19/11/02 02:19:30 INFO util.AppendUtils: Appending to directory order_items
19/11/02 02:19:30 INFO util.AppendUtils: Using found partition 4
[cloudera@quickstart ~]$ hdfs dfs -ls /user/training/sqoop_import/retail_db/order_itemsFound 9 items
-rw-r--r-- 1 cloudera supergroup 0 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/_SUCCESS
-rw-r--r-- 1 cloudera supergroup 1303818 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/part-m-00000
-rw-r--r-- 1 cloudera supergroup 1343222 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/part-m-00001
-rw-r--r-- 1 cloudera supergroup 1371917 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/part-m-00002
-rw-r--r-- 1 cloudera supergroup 1389923 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/part-m-00003
-rw-r--r-- 1 cloudera cloudera 1303818 2019-11-02 02:19 /user/training/sqoop_import/retail_db/order_items/part-m-00004
-rw-r--r-- 1 cloudera cloudera 1343222 2019-11-02 02:19 /user/training/sqoop_import/retail_db/order_items/part-m-00005
-rw-r--r-- 1 cloudera cloudera 1371917 2019-11-02 02:19 /user/training/sqoop_import/retail_db/order_items/part-m-00006
-rw-r--r-- 1 cloudera cloudera 1389923 2019-11-02 02:19 /user/training/sqoop_import/retail_db/order_items/part-m-00007
[cloudera@quickstart ~]$ -rw-r--r-- 1 cloudera supergroup 1389923 2019-11-02 02:15 /user/training/sqoop_import/retail_db/order_items/part-m-00003
bash: -rw-r--r--: command not found
[cloudera@quickstart ~]$ sqoop import --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera --table order_items --warehouse-dir /user/training/sqoop_import/retail_db --delete-target-dir
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 02:21:45 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 02:21:45 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 02:21:46 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/11/02 02:21:46 INFO tool.CodeGenTool: Beginning code generation
19/11/02 02:21:46 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 02:21:47 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 02:21:47 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/6289265f19a88a163a949ccdae0e4609/order_items.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/11/02 02:21:50 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/6289265f19a88a163a949ccdae0e4609/order_items.jar
19/11/02 02:21:51 INFO tool.ImportTool: Destination directory /user/training/sqoop_import/retail_db/order_items deleted.
19/11/02 02:21:51 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/11/02 02:21:51 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/11/02 02:21:51 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/11/02 02:21:51 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/11/02 02:21:51 INFO mapreduce.ImportJobBase: Beginning import of order_items
19/11/02 02:21:51 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
19/11/02 02:21:51 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/11/02 02:21:51 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/11/02 02:21:52 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/11/02 02:21:58 INFO db.DBInputFormat: Using read commited transaction isolation
19/11/02 02:21:58 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`order_item_id`), MAX(`order_item_id`) FROM `order_items`
19/11/02 02:21:58 INFO db.IntegerSplitter: Split size: 43049; Num splits: 4 from: 1 to: 172198
19/11/02 02:21:58 INFO mapreduce.JobSubmitter: number of splits:4
19/11/02 02:21:58 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1572629054486_0004
19/11/02 02:21:59 INFO impl.YarnClientImpl: Submitted application application_1572629054486_0004
19/11/02 02:21:59 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1572629054486_0004/
19/11/02 02:21:59 INFO mapreduce.Job: Running job: job_1572629054486_0004
19/11/02 02:22:09 INFO mapreduce.Job: Job job_1572629054486_0004 running in uber mode : false
19/11/02 02:22:09 INFO mapreduce.Job: map 0% reduce 0%
19/11/02 02:22:30 INFO mapreduce.Job: map 25% reduce 0%
19/11/02 02:22:31 INFO mapreduce.Job: map 50% reduce 0%
19/11/02 02:22:34 INFO mapreduce.Job: map 100% reduce 0%
19/11/02 02:22:36 INFO mapreduce.Job: Job job_1572629054486_0004 completed successfully
19/11/02 02:22:36 INFO mapreduce.Job: Counters: 31
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=686024
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=512
HDFS: Number of bytes written=5408880
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Killed map tasks=1
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=80397
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=80397
Total vcore-milliseconds taken by all map tasks=80397
Total megabyte-milliseconds taken by all map tasks=82326528
Map-Reduce Framework
Map input records=172198
Map output records=172198
Input split bytes=512
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=658
CPU time spent (ms)=24240
Physical memory (bytes) snapshot=942854144
Virtual memory (bytes) snapshot=6264147968
Total committed heap usage (bytes)=618659840
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=5408880
19/11/02 02:22:36 INFO mapreduce.ImportJobBase: Transferred 5.1583 MB in 44.2065 seconds (119.4871 KB/sec)
19/11/02 02:22:36 INFO mapreduce.ImportJobBase: Retrieved 172198 records.
[cloudera@quickstart ~]$ hdfs dfs -ls /user/training/sqoop_import/retail_db/order_itemsFound 5 items
-rw-r--r-- 1 cloudera supergroup 0 2019-11-02 02:22 /user/training/sqoop_import/retail_db/order_items/_SUCCESS
-rw-r--r-- 1 cloudera supergroup 1303818 2019-11-02 02:22 /user/training/sqoop_import/retail_db/order_items/part-m-00000
-rw-r--r-- 1 cloudera supergroup 1343222 2019-11-02 02:22 /user/training/sqoop_import/retail_db/order_items/part-m-00001
-rw-r--r-- 1 cloudera supergroup 1371917 2019-11-02 02:22 /user/training/sqoop_import/retail_db/order_items/part-m-00002
-rw-r--r-- 1 cloudera supergroup 1389923 2019-11-02 02:22 /user/training/sqoop_import/retail_db/order_items/part-m-00003
[cloudera@quickstart ~]$
We can import data and we cam append or overwrite the content
of the table
No comments:
Post a Comment