[cloudera@quickstart ~]$ sqoop eval --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera -e "DESCRIBE order_items"
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 03:29:30 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 03:29:30 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 03:29:30 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
---------------------------------------------------------------------------------------------------------
| Field | Type | Null | Key | Default | Extra |
---------------------------------------------------------------------------------------------------------
| order_item_id | int(11) | NO | PRI | (null) | auto_increment |
| order_item_order_id | int(11) | NO | | (null) | |
| order_item_product_id | int(11) | NO | | (null) | |
| order_item_quantity | tinyint(4) | NO | | (null) | |
| order_item_subtotal | float | NO | | (null) | |
| order_item_product_price | float | NO | | (null) | |
---------------------------------------------------------------------------------------------------------
[cloudera@quickstart ~]$ sqoop import --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera --table customers --warehouse-dir /user/training/sqoop_import/retail_db --delete-target-dir --num-mappers 8 --compress --compression-codec org.apache.hadoop.io.compress.SnappyCodec --columns customer_id,customer_fname,customer_lname,customer_street,customer_city,customer_state,customer_zipcode
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 03:34:08 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 03:34:08 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 03:34:08 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/11/02 03:34:08 INFO tool.CodeGenTool: Beginning code generation
19/11/02 03:34:09 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `customers` AS t LIMIT 1
19/11/02 03:34:09 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `customers` AS t LIMIT 1
19/11/02 03:34:09 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/c3dd5434c97c5041b286f69c193eb43f/customers.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/11/02 03:34:12 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/c3dd5434c97c5041b286f69c193eb43f/customers.jar
19/11/02 03:34:14 INFO tool.ImportTool: Destination directory /user/training/sqoop_import/retail_db/customers is not present, hence not deleting.
19/11/02 03:34:14 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/11/02 03:34:14 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/11/02 03:34:14 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/11/02 03:34:14 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/11/02 03:34:14 INFO mapreduce.ImportJobBase: Beginning import of customers
19/11/02 03:34:14 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
19/11/02 03:34:14 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/11/02 03:34:14 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/11/02 03:34:14 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/11/02 03:34:20 INFO db.DBInputFormat: Using read commited transaction isolation
19/11/02 03:34:20 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`customer_id`), MAX(`customer_id`) FROM `customers`
19/11/02 03:34:20 INFO db.IntegerSplitter: Split size: 1554; Num splits: 8 from: 1 to: 12435
19/11/02 03:34:21 INFO mapreduce.JobSubmitter: number of splits:8
19/11/02 03:34:21 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1572629054486_0009
19/11/02 03:34:22 INFO impl.YarnClientImpl: Submitted application application_1572629054486_0009
19/11/02 03:34:22 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1572629054486_0009/
19/11/02 03:34:22 INFO mapreduce.Job: Running job: job_1572629054486_0009
19/11/02 03:34:32 INFO mapreduce.Job: Job job_1572629054486_0009 running in uber mode : false
19/11/02 03:34:32 INFO mapreduce.Job: map 0% reduce 0%
19/11/02 03:35:13 INFO mapreduce.Job: map 13% reduce 0%
19/11/02 03:35:17 INFO mapreduce.Job: map 25% reduce 0%
19/11/02 03:35:24 INFO mapreduce.Job: map 75% reduce 0%
19/11/02 03:35:35 INFO mapreduce.Job: map 100% reduce 0%
19/11/02 03:35:36 INFO mapreduce.Job: Job job_1572629054486_0009 completed successfully
19/11/02 03:35:36 INFO mapreduce.Job: Counters: 31
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=1375368
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=977
HDFS: Number of bytes written=407586
HDFS: Number of read operations=32
HDFS: Number of large read operations=0
HDFS: Number of write operations=16
Job Counters
Killed map tasks=1
Launched map tasks=8
Other local map tasks=8
Total time spent by all maps in occupied slots (ms)=291969
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=291969
Total vcore-milliseconds taken by all map tasks=291969
Total megabyte-milliseconds taken by all map tasks=298976256
Map-Reduce Framework
Map input records=12435
Map output records=12435
Input split bytes=977
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=2088
CPU time spent (ms)=19080
Physical memory (bytes) snapshot=1425698816
Virtual memory (bytes) snapshot=12597407744
Total committed heap usage (bytes)=1224736768
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=407586
19/11/02 03:35:36 INFO mapreduce.ImportJobBase: Transferred 398.0332 KB in 81.6043 seconds (4.8776 KB/sec)
19/11/02 03:35:36 INFO mapreduce.ImportJobBase: Retrieved 12435 records.
[cloudera@quickstart ~]$ sqoop eval --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera -e "DESCRIBE customers"Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 03:37:41 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 03:37:41 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 03:37:42 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
---------------------------------------------------------------------------------------------------------
| Field | Type | Null | Key | Default | Extra |
---------------------------------------------------------------------------------------------------------
| customer_id | int(11) | NO | PRI | (null) | auto_increment |
| customer_fname | varchar(45) | NO | | (null) | |
| customer_lname | varchar(45) | NO | | (null) | |
| customer_email | varchar(45) | NO | | (null) | |
| customer_password | varchar(45) | NO | | (null) | |
| customer_street | varchar(255) | NO | | (null) | |
| customer_city | varchar(45) | NO | | (null) | |
| customer_state | varchar(45) | NO | | (null) | |
| customer_zipcode | varchar(45) | NO | | (null) | |
---------------------------------------------------------------------------------------------------------
[cloudera@quickstart ~]$ sqoop import --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera --table order_items --warehouse-dir /user/training/sqoop_import/retail_db --delete-target-dir
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 03:39:30 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 03:39:30 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 03:39:30 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/11/02 03:39:31 INFO tool.CodeGenTool: Beginning code generation
19/11/02 03:39:31 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 03:39:31 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 03:39:31 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/71e2019696ef6afae0527fbf3efbba2d/order_items.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/11/02 03:39:34 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/71e2019696ef6afae0527fbf3efbba2d/order_items.jar
19/11/02 03:39:36 INFO tool.ImportTool: Destination directory /user/training/sqoop_import/retail_db/order_items deleted.
19/11/02 03:39:36 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/11/02 03:39:36 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/11/02 03:39:36 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/11/02 03:39:36 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/11/02 03:39:36 INFO mapreduce.ImportJobBase: Beginning import of order_items
19/11/02 03:39:36 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
19/11/02 03:39:36 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/11/02 03:39:36 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/11/02 03:39:37 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/11/02 03:39:44 INFO db.DBInputFormat: Using read commited transaction isolation
19/11/02 03:39:44 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`order_item_id`), MAX(`order_item_id`) FROM `order_items`
19/11/02 03:39:44 INFO db.IntegerSplitter: Split size: 43049; Num splits: 4 from: 1 to: 172198
19/11/02 03:39:45 INFO mapreduce.JobSubmitter: number of splits:4
19/11/02 03:39:45 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1572629054486_0010
19/11/02 03:39:46 INFO impl.YarnClientImpl: Submitted application application_1572629054486_0010
19/11/02 03:39:46 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1572629054486_0010/
19/11/02 03:39:46 INFO mapreduce.Job: Running job: job_1572629054486_0010
19/11/02 03:39:56 INFO mapreduce.Job: Job job_1572629054486_0010 running in uber mode : false
19/11/02 03:39:56 INFO mapreduce.Job: map 0% reduce 0%
19/11/02 03:40:23 INFO mapreduce.Job: map 25% reduce 0%
19/11/02 03:40:27 INFO mapreduce.Job: map 100% reduce 0%
19/11/02 03:40:29 INFO mapreduce.Job: Job job_1572629054486_0010 completed successfully
19/11/02 03:40:29 INFO mapreduce.Job: Counters: 31
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=686024
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=512
HDFS: Number of bytes written=5408880
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Killed map tasks=1
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=103289
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=103289
Total vcore-milliseconds taken by all map tasks=103289
Total megabyte-milliseconds taken by all map tasks=105767936
Map-Reduce Framework
Map input records=172198
Map output records=172198
Input split bytes=512
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=1044
CPU time spent (ms)=28280
Physical memory (bytes) snapshot=996171776
Virtual memory (bytes) snapshot=6293839872
Total committed heap usage (bytes)=698875904
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=5408880
19/11/02 03:40:29 INFO mapreduce.ImportJobBase: Transferred 5.1583 MB in 52.8021 seconds (100.036 KB/sec)
19/11/02 03:40:29 INFO mapreduce.ImportJobBase: Retrieved 172198 records.
[cloudera@quickstart ~]$ hdfs dfs -ls /user/training/sqoop_import/retail_db/order_itemsFound 5 items
-rw-r--r-- 1 cloudera supergroup 0 2019-11-02 03:40 /user/training/sqoop_import/retail_db/order_items/_SUCCESS
-rw-r--r-- 1 cloudera supergroup 1303818 2019-11-02 03:40 /user/training/sqoop_import/retail_db/order_items/part-m-00000
-rw-r--r-- 1 cloudera supergroup 1343222 2019-11-02 03:40 /user/training/sqoop_import/retail_db/order_items/part-m-00001
-rw-r--r-- 1 cloudera supergroup 1371917 2019-11-02 03:40 /user/training/sqoop_import/retail_db/order_items/part-m-00002
-rw-r--r-- 1 cloudera supergroup 1389923 2019-11-02 03:40 /user/training/sqoop_import/retail_db/order_items/part-m-00003
[cloudera@quickstart ~]$ sqoop import --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera --table order_items --warehouse-dir /user/training/sqoop_import/retail_db --delete-target-dir --boundary-query "SELECT 1,172198"
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 03:41:32 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 03:41:32 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 03:41:32 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/11/02 03:41:32 INFO tool.CodeGenTool: Beginning code generation
19/11/02 03:41:33 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 03:41:33 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 03:41:33 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/312090c24d79c9bbb4e59629b40ca110/order_items.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/11/02 03:41:36 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/312090c24d79c9bbb4e59629b40ca110/order_items.jar
19/11/02 03:41:38 INFO tool.ImportTool: Destination directory /user/training/sqoop_import/retail_db/order_items deleted.
19/11/02 03:41:38 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/11/02 03:41:38 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/11/02 03:41:38 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/11/02 03:41:38 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/11/02 03:41:38 INFO mapreduce.ImportJobBase: Beginning import of order_items
19/11/02 03:41:38 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
19/11/02 03:41:38 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/11/02 03:41:38 WARN db.DataDrivenDBInputFormat: Could not find $CONDITIONS token in query: SELECT 1,172198; splits may not partition data.
19/11/02 03:41:38 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/11/02 03:41:39 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/11/02 03:41:45 INFO db.DBInputFormat: Using read commited transaction isolation
19/11/02 03:41:45 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT 1,172198
19/11/02 03:41:45 INFO db.IntegerSplitter: Split size: 43049; Num splits: 4 from: 1 to: 172198
19/11/02 03:41:46 INFO mapreduce.JobSubmitter: number of splits:4
19/11/02 03:41:46 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1572629054486_0011
19/11/02 03:41:47 INFO impl.YarnClientImpl: Submitted application application_1572629054486_0011
19/11/02 03:41:47 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1572629054486_0011/
19/11/02 03:41:47 INFO mapreduce.Job: Running job: job_1572629054486_0011
19/11/02 03:41:56 INFO mapreduce.Job: Job job_1572629054486_0011 running in uber mode : false
19/11/02 03:41:56 INFO mapreduce.Job: map 0% reduce 0%
19/11/02 03:42:18 INFO mapreduce.Job: map 25% reduce 0%
19/11/02 03:42:20 INFO mapreduce.Job: map 50% reduce 0%
19/11/02 03:42:22 INFO mapreduce.Job: map 75% reduce 0%
19/11/02 03:42:23 INFO mapreduce.Job: map 100% reduce 0%
19/11/02 03:42:25 INFO mapreduce.Job: Job job_1572629054486_0011 completed successfully
19/11/02 03:42:25 INFO mapreduce.Job: Counters: 31
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=687404
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=512
HDFS: Number of bytes written=5408880
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Killed map tasks=1
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=86566
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=86566
Total vcore-milliseconds taken by all map tasks=86566
Total megabyte-milliseconds taken by all map tasks=88643584
Map-Reduce Framework
Map input records=172198
Map output records=172198
Input split bytes=512
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=753
CPU time spent (ms)=25520
Physical memory (bytes) snapshot=963543040
Virtual memory (bytes) snapshot=6291369984
Total committed heap usage (bytes)=651689984
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=5408880
19/11/02 03:42:25 INFO mapreduce.ImportJobBase: Transferred 5.1583 MB in 46.3702 seconds (113.9117 KB/sec)
19/11/02 03:42:25 INFO mapreduce.ImportJobBase: Retrieved 172198 records.
[cloudera@quickstart ~]$ hdfs dfs -ls /user/training/sqoop_import/retail_db/order_itemsFound 5 items
-rw-r--r-- 1 cloudera supergroup 0 2019-11-02 03:42 /user/training/sqoop_import/retail_db/order_items/_SUCCESS
-rw-r--r-- 1 cloudera supergroup 1303818 2019-11-02 03:42 /user/training/sqoop_import/retail_db/order_items/part-m-00000
-rw-r--r-- 1 cloudera supergroup 1343222 2019-11-02 03:42 /user/training/sqoop_import/retail_db/order_items/part-m-00001
-rw-r--r-- 1 cloudera supergroup 1371917 2019-11-02 03:42 /user/training/sqoop_import/retail_db/order_items/part-m-00002
-rw-r--r-- 1 cloudera supergroup 1389923 2019-11-02 03:42 /user/training/sqoop_import/retail_db/order_items/part-m-00003
[cloudera@quickstart ~]$ sqoop import --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera --table order_items --warehouse-dir /user/training/sqoop_import/retail_db --delete-target-dir --boundary-query "SELECT 1,50000"
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 03:42:53 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 03:42:53 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 03:42:53 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/11/02 03:42:53 INFO tool.CodeGenTool: Beginning code generation
19/11/02 03:42:54 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 03:42:54 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 03:42:54 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/53ea1935edc38ef18c85d22bad5f437b/order_items.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/11/02 03:42:57 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/53ea1935edc38ef18c85d22bad5f437b/order_items.jar
19/11/02 03:42:59 INFO tool.ImportTool: Destination directory /user/training/sqoop_import/retail_db/order_items deleted.
19/11/02 03:42:59 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/11/02 03:42:59 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/11/02 03:42:59 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/11/02 03:42:59 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/11/02 03:42:59 INFO mapreduce.ImportJobBase: Beginning import of order_items
19/11/02 03:42:59 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
19/11/02 03:42:59 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/11/02 03:42:59 WARN db.DataDrivenDBInputFormat: Could not find $CONDITIONS token in query: SELECT 1,50000; splits may not partition data.
19/11/02 03:42:59 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/11/02 03:42:59 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/11/02 03:43:06 INFO db.DBInputFormat: Using read commited transaction isolation
19/11/02 03:43:06 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT 1,50000
19/11/02 03:43:06 INFO db.IntegerSplitter: Split size: 12499; Num splits: 4 from: 1 to: 50000
19/11/02 03:43:06 INFO mapreduce.JobSubmitter: number of splits:4
19/11/02 03:43:06 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1572629054486_0012
19/11/02 03:43:07 INFO impl.YarnClientImpl: Submitted application application_1572629054486_0012
19/11/02 03:43:07 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1572629054486_0012/
19/11/02 03:43:07 INFO mapreduce.Job: Running job: job_1572629054486_0012
19/11/02 03:43:16 INFO mapreduce.Job: Job job_1572629054486_0012 running in uber mode : false
19/11/02 03:43:16 INFO mapreduce.Job: map 0% reduce 0%
19/11/02 03:43:37 INFO mapreduce.Job: map 25% reduce 0%
19/11/02 03:43:39 INFO mapreduce.Job: map 75% reduce 0%
19/11/02 03:43:43 INFO mapreduce.Job: map 100% reduce 0%
19/11/02 03:43:44 INFO mapreduce.Job: Job job_1572629054486_0012 completed successfully
19/11/02 03:43:44 INFO mapreduce.Job: Counters: 31
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=687396
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=509
HDFS: Number of bytes written=1520520
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Killed map tasks=1
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=72377
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=72377
Total vcore-milliseconds taken by all map tasks=72377
Total megabyte-milliseconds taken by all map tasks=74114048
Map-Reduce Framework
Map input records=50000
Map output records=50000
Input split bytes=509
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=399
CPU time spent (ms)=19950
Physical memory (bytes) snapshot=860155904
Virtual memory (bytes) snapshot=6283833344
Total committed heap usage (bytes)=642777088
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=1520520
19/11/02 03:43:44 INFO mapreduce.ImportJobBase: Transferred 1.4501 MB in 45.192 seconds (32.8572 KB/sec)
19/11/02 03:43:44 INFO mapreduce.ImportJobBase: Retrieved 50000 records.
[cloudera@quickstart ~]$ hdfs dfs -ls /user/training/sqoop_import/retail_db/order_itemsFound 5 items
-rw-r--r-- 1 cloudera supergroup 0 2019-11-02 03:43 /user/training/sqoop_import/retail_db/order_items/_SUCCESS
-rw-r--r-- 1 cloudera supergroup 363500 2019-11-02 03:43 /user/training/sqoop_import/retail_db/order_items/part-m-00000
-rw-r--r-- 1 cloudera supergroup 377289 2019-11-02 03:43 /user/training/sqoop_import/retail_db/order_items/part-m-00001
-rw-r--r-- 1 cloudera supergroup 389972 2019-11-02 03:43 /user/training/sqoop_import/retail_db/order_items/part-m-00002
-rw-r--r-- 1 cloudera supergroup 389759 2019-11-02 03:43 /user/training/sqoop_import/retail_db/order_items/part-m-00003
[cloudera@quickstart ~]$
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 03:29:30 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 03:29:30 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 03:29:30 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
---------------------------------------------------------------------------------------------------------
| Field | Type | Null | Key | Default | Extra |
---------------------------------------------------------------------------------------------------------
| order_item_id | int(11) | NO | PRI | (null) | auto_increment |
| order_item_order_id | int(11) | NO | | (null) | |
| order_item_product_id | int(11) | NO | | (null) | |
| order_item_quantity | tinyint(4) | NO | | (null) | |
| order_item_subtotal | float | NO | | (null) | |
| order_item_product_price | float | NO | | (null) | |
---------------------------------------------------------------------------------------------------------
[cloudera@quickstart ~]$ sqoop import --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera --table customers --warehouse-dir /user/training/sqoop_import/retail_db --delete-target-dir --num-mappers 8 --compress --compression-codec org.apache.hadoop.io.compress.SnappyCodec --columns customer_id,customer_fname,customer_lname,customer_street,customer_city,customer_state,customer_zipcode
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 03:34:08 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 03:34:08 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 03:34:08 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/11/02 03:34:08 INFO tool.CodeGenTool: Beginning code generation
19/11/02 03:34:09 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `customers` AS t LIMIT 1
19/11/02 03:34:09 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `customers` AS t LIMIT 1
19/11/02 03:34:09 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/c3dd5434c97c5041b286f69c193eb43f/customers.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/11/02 03:34:12 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/c3dd5434c97c5041b286f69c193eb43f/customers.jar
19/11/02 03:34:14 INFO tool.ImportTool: Destination directory /user/training/sqoop_import/retail_db/customers is not present, hence not deleting.
19/11/02 03:34:14 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/11/02 03:34:14 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/11/02 03:34:14 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/11/02 03:34:14 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/11/02 03:34:14 INFO mapreduce.ImportJobBase: Beginning import of customers
19/11/02 03:34:14 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
19/11/02 03:34:14 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/11/02 03:34:14 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/11/02 03:34:14 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/11/02 03:34:20 INFO db.DBInputFormat: Using read commited transaction isolation
19/11/02 03:34:20 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`customer_id`), MAX(`customer_id`) FROM `customers`
19/11/02 03:34:20 INFO db.IntegerSplitter: Split size: 1554; Num splits: 8 from: 1 to: 12435
19/11/02 03:34:21 INFO mapreduce.JobSubmitter: number of splits:8
19/11/02 03:34:21 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1572629054486_0009
19/11/02 03:34:22 INFO impl.YarnClientImpl: Submitted application application_1572629054486_0009
19/11/02 03:34:22 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1572629054486_0009/
19/11/02 03:34:22 INFO mapreduce.Job: Running job: job_1572629054486_0009
19/11/02 03:34:32 INFO mapreduce.Job: Job job_1572629054486_0009 running in uber mode : false
19/11/02 03:34:32 INFO mapreduce.Job: map 0% reduce 0%
19/11/02 03:35:13 INFO mapreduce.Job: map 13% reduce 0%
19/11/02 03:35:17 INFO mapreduce.Job: map 25% reduce 0%
19/11/02 03:35:24 INFO mapreduce.Job: map 75% reduce 0%
19/11/02 03:35:35 INFO mapreduce.Job: map 100% reduce 0%
19/11/02 03:35:36 INFO mapreduce.Job: Job job_1572629054486_0009 completed successfully
19/11/02 03:35:36 INFO mapreduce.Job: Counters: 31
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=1375368
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=977
HDFS: Number of bytes written=407586
HDFS: Number of read operations=32
HDFS: Number of large read operations=0
HDFS: Number of write operations=16
Job Counters
Killed map tasks=1
Launched map tasks=8
Other local map tasks=8
Total time spent by all maps in occupied slots (ms)=291969
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=291969
Total vcore-milliseconds taken by all map tasks=291969
Total megabyte-milliseconds taken by all map tasks=298976256
Map-Reduce Framework
Map input records=12435
Map output records=12435
Input split bytes=977
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=2088
CPU time spent (ms)=19080
Physical memory (bytes) snapshot=1425698816
Virtual memory (bytes) snapshot=12597407744
Total committed heap usage (bytes)=1224736768
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=407586
19/11/02 03:35:36 INFO mapreduce.ImportJobBase: Transferred 398.0332 KB in 81.6043 seconds (4.8776 KB/sec)
19/11/02 03:35:36 INFO mapreduce.ImportJobBase: Retrieved 12435 records.
[cloudera@quickstart ~]$ sqoop eval --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera -e "DESCRIBE customers"Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 03:37:41 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 03:37:41 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 03:37:42 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
---------------------------------------------------------------------------------------------------------
| Field | Type | Null | Key | Default | Extra |
---------------------------------------------------------------------------------------------------------
| customer_id | int(11) | NO | PRI | (null) | auto_increment |
| customer_fname | varchar(45) | NO | | (null) | |
| customer_lname | varchar(45) | NO | | (null) | |
| customer_email | varchar(45) | NO | | (null) | |
| customer_password | varchar(45) | NO | | (null) | |
| customer_street | varchar(255) | NO | | (null) | |
| customer_city | varchar(45) | NO | | (null) | |
| customer_state | varchar(45) | NO | | (null) | |
| customer_zipcode | varchar(45) | NO | | (null) | |
---------------------------------------------------------------------------------------------------------
[cloudera@quickstart ~]$ sqoop import --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera --table order_items --warehouse-dir /user/training/sqoop_import/retail_db --delete-target-dir
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 03:39:30 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 03:39:30 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 03:39:30 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/11/02 03:39:31 INFO tool.CodeGenTool: Beginning code generation
19/11/02 03:39:31 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 03:39:31 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 03:39:31 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/71e2019696ef6afae0527fbf3efbba2d/order_items.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/11/02 03:39:34 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/71e2019696ef6afae0527fbf3efbba2d/order_items.jar
19/11/02 03:39:36 INFO tool.ImportTool: Destination directory /user/training/sqoop_import/retail_db/order_items deleted.
19/11/02 03:39:36 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/11/02 03:39:36 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/11/02 03:39:36 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/11/02 03:39:36 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/11/02 03:39:36 INFO mapreduce.ImportJobBase: Beginning import of order_items
19/11/02 03:39:36 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
19/11/02 03:39:36 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/11/02 03:39:36 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/11/02 03:39:37 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/11/02 03:39:44 INFO db.DBInputFormat: Using read commited transaction isolation
19/11/02 03:39:44 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`order_item_id`), MAX(`order_item_id`) FROM `order_items`
19/11/02 03:39:44 INFO db.IntegerSplitter: Split size: 43049; Num splits: 4 from: 1 to: 172198
19/11/02 03:39:45 INFO mapreduce.JobSubmitter: number of splits:4
19/11/02 03:39:45 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1572629054486_0010
19/11/02 03:39:46 INFO impl.YarnClientImpl: Submitted application application_1572629054486_0010
19/11/02 03:39:46 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1572629054486_0010/
19/11/02 03:39:46 INFO mapreduce.Job: Running job: job_1572629054486_0010
19/11/02 03:39:56 INFO mapreduce.Job: Job job_1572629054486_0010 running in uber mode : false
19/11/02 03:39:56 INFO mapreduce.Job: map 0% reduce 0%
19/11/02 03:40:23 INFO mapreduce.Job: map 25% reduce 0%
19/11/02 03:40:27 INFO mapreduce.Job: map 100% reduce 0%
19/11/02 03:40:29 INFO mapreduce.Job: Job job_1572629054486_0010 completed successfully
19/11/02 03:40:29 INFO mapreduce.Job: Counters: 31
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=686024
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=512
HDFS: Number of bytes written=5408880
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Killed map tasks=1
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=103289
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=103289
Total vcore-milliseconds taken by all map tasks=103289
Total megabyte-milliseconds taken by all map tasks=105767936
Map-Reduce Framework
Map input records=172198
Map output records=172198
Input split bytes=512
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=1044
CPU time spent (ms)=28280
Physical memory (bytes) snapshot=996171776
Virtual memory (bytes) snapshot=6293839872
Total committed heap usage (bytes)=698875904
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=5408880
19/11/02 03:40:29 INFO mapreduce.ImportJobBase: Transferred 5.1583 MB in 52.8021 seconds (100.036 KB/sec)
19/11/02 03:40:29 INFO mapreduce.ImportJobBase: Retrieved 172198 records.
[cloudera@quickstart ~]$ hdfs dfs -ls /user/training/sqoop_import/retail_db/order_itemsFound 5 items
-rw-r--r-- 1 cloudera supergroup 0 2019-11-02 03:40 /user/training/sqoop_import/retail_db/order_items/_SUCCESS
-rw-r--r-- 1 cloudera supergroup 1303818 2019-11-02 03:40 /user/training/sqoop_import/retail_db/order_items/part-m-00000
-rw-r--r-- 1 cloudera supergroup 1343222 2019-11-02 03:40 /user/training/sqoop_import/retail_db/order_items/part-m-00001
-rw-r--r-- 1 cloudera supergroup 1371917 2019-11-02 03:40 /user/training/sqoop_import/retail_db/order_items/part-m-00002
-rw-r--r-- 1 cloudera supergroup 1389923 2019-11-02 03:40 /user/training/sqoop_import/retail_db/order_items/part-m-00003
[cloudera@quickstart ~]$ sqoop import --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera --table order_items --warehouse-dir /user/training/sqoop_import/retail_db --delete-target-dir --boundary-query "SELECT 1,172198"
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 03:41:32 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 03:41:32 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 03:41:32 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/11/02 03:41:32 INFO tool.CodeGenTool: Beginning code generation
19/11/02 03:41:33 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 03:41:33 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 03:41:33 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/312090c24d79c9bbb4e59629b40ca110/order_items.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/11/02 03:41:36 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/312090c24d79c9bbb4e59629b40ca110/order_items.jar
19/11/02 03:41:38 INFO tool.ImportTool: Destination directory /user/training/sqoop_import/retail_db/order_items deleted.
19/11/02 03:41:38 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/11/02 03:41:38 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/11/02 03:41:38 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/11/02 03:41:38 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/11/02 03:41:38 INFO mapreduce.ImportJobBase: Beginning import of order_items
19/11/02 03:41:38 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
19/11/02 03:41:38 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/11/02 03:41:38 WARN db.DataDrivenDBInputFormat: Could not find $CONDITIONS token in query: SELECT 1,172198; splits may not partition data.
19/11/02 03:41:38 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/11/02 03:41:39 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/11/02 03:41:45 INFO db.DBInputFormat: Using read commited transaction isolation
19/11/02 03:41:45 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT 1,172198
19/11/02 03:41:45 INFO db.IntegerSplitter: Split size: 43049; Num splits: 4 from: 1 to: 172198
19/11/02 03:41:46 INFO mapreduce.JobSubmitter: number of splits:4
19/11/02 03:41:46 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1572629054486_0011
19/11/02 03:41:47 INFO impl.YarnClientImpl: Submitted application application_1572629054486_0011
19/11/02 03:41:47 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1572629054486_0011/
19/11/02 03:41:47 INFO mapreduce.Job: Running job: job_1572629054486_0011
19/11/02 03:41:56 INFO mapreduce.Job: Job job_1572629054486_0011 running in uber mode : false
19/11/02 03:41:56 INFO mapreduce.Job: map 0% reduce 0%
19/11/02 03:42:18 INFO mapreduce.Job: map 25% reduce 0%
19/11/02 03:42:20 INFO mapreduce.Job: map 50% reduce 0%
19/11/02 03:42:22 INFO mapreduce.Job: map 75% reduce 0%
19/11/02 03:42:23 INFO mapreduce.Job: map 100% reduce 0%
19/11/02 03:42:25 INFO mapreduce.Job: Job job_1572629054486_0011 completed successfully
19/11/02 03:42:25 INFO mapreduce.Job: Counters: 31
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=687404
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=512
HDFS: Number of bytes written=5408880
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Killed map tasks=1
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=86566
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=86566
Total vcore-milliseconds taken by all map tasks=86566
Total megabyte-milliseconds taken by all map tasks=88643584
Map-Reduce Framework
Map input records=172198
Map output records=172198
Input split bytes=512
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=753
CPU time spent (ms)=25520
Physical memory (bytes) snapshot=963543040
Virtual memory (bytes) snapshot=6291369984
Total committed heap usage (bytes)=651689984
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=5408880
19/11/02 03:42:25 INFO mapreduce.ImportJobBase: Transferred 5.1583 MB in 46.3702 seconds (113.9117 KB/sec)
19/11/02 03:42:25 INFO mapreduce.ImportJobBase: Retrieved 172198 records.
[cloudera@quickstart ~]$ hdfs dfs -ls /user/training/sqoop_import/retail_db/order_itemsFound 5 items
-rw-r--r-- 1 cloudera supergroup 0 2019-11-02 03:42 /user/training/sqoop_import/retail_db/order_items/_SUCCESS
-rw-r--r-- 1 cloudera supergroup 1303818 2019-11-02 03:42 /user/training/sqoop_import/retail_db/order_items/part-m-00000
-rw-r--r-- 1 cloudera supergroup 1343222 2019-11-02 03:42 /user/training/sqoop_import/retail_db/order_items/part-m-00001
-rw-r--r-- 1 cloudera supergroup 1371917 2019-11-02 03:42 /user/training/sqoop_import/retail_db/order_items/part-m-00002
-rw-r--r-- 1 cloudera supergroup 1389923 2019-11-02 03:42 /user/training/sqoop_import/retail_db/order_items/part-m-00003
[cloudera@quickstart ~]$ sqoop import --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username root --password cloudera --table order_items --warehouse-dir /user/training/sqoop_import/retail_db --delete-target-dir --boundary-query "SELECT 1,50000"
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/11/02 03:42:53 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
19/11/02 03:42:53 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/02 03:42:53 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/11/02 03:42:53 INFO tool.CodeGenTool: Beginning code generation
19/11/02 03:42:54 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 03:42:54 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `order_items` AS t LIMIT 1
19/11/02 03:42:54 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/53ea1935edc38ef18c85d22bad5f437b/order_items.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/11/02 03:42:57 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/53ea1935edc38ef18c85d22bad5f437b/order_items.jar
19/11/02 03:42:59 INFO tool.ImportTool: Destination directory /user/training/sqoop_import/retail_db/order_items deleted.
19/11/02 03:42:59 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/11/02 03:42:59 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/11/02 03:42:59 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/11/02 03:42:59 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/11/02 03:42:59 INFO mapreduce.ImportJobBase: Beginning import of order_items
19/11/02 03:42:59 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
19/11/02 03:42:59 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/11/02 03:42:59 WARN db.DataDrivenDBInputFormat: Could not find $CONDITIONS token in query: SELECT 1,50000; splits may not partition data.
19/11/02 03:42:59 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/11/02 03:42:59 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/11/02 03:43:06 INFO db.DBInputFormat: Using read commited transaction isolation
19/11/02 03:43:06 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT 1,50000
19/11/02 03:43:06 INFO db.IntegerSplitter: Split size: 12499; Num splits: 4 from: 1 to: 50000
19/11/02 03:43:06 INFO mapreduce.JobSubmitter: number of splits:4
19/11/02 03:43:06 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1572629054486_0012
19/11/02 03:43:07 INFO impl.YarnClientImpl: Submitted application application_1572629054486_0012
19/11/02 03:43:07 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1572629054486_0012/
19/11/02 03:43:07 INFO mapreduce.Job: Running job: job_1572629054486_0012
19/11/02 03:43:16 INFO mapreduce.Job: Job job_1572629054486_0012 running in uber mode : false
19/11/02 03:43:16 INFO mapreduce.Job: map 0% reduce 0%
19/11/02 03:43:37 INFO mapreduce.Job: map 25% reduce 0%
19/11/02 03:43:39 INFO mapreduce.Job: map 75% reduce 0%
19/11/02 03:43:43 INFO mapreduce.Job: map 100% reduce 0%
19/11/02 03:43:44 INFO mapreduce.Job: Job job_1572629054486_0012 completed successfully
19/11/02 03:43:44 INFO mapreduce.Job: Counters: 31
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=687396
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=509
HDFS: Number of bytes written=1520520
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Killed map tasks=1
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=72377
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=72377
Total vcore-milliseconds taken by all map tasks=72377
Total megabyte-milliseconds taken by all map tasks=74114048
Map-Reduce Framework
Map input records=50000
Map output records=50000
Input split bytes=509
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=399
CPU time spent (ms)=19950
Physical memory (bytes) snapshot=860155904
Virtual memory (bytes) snapshot=6283833344
Total committed heap usage (bytes)=642777088
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=1520520
19/11/02 03:43:44 INFO mapreduce.ImportJobBase: Transferred 1.4501 MB in 45.192 seconds (32.8572 KB/sec)
19/11/02 03:43:44 INFO mapreduce.ImportJobBase: Retrieved 50000 records.
[cloudera@quickstart ~]$ hdfs dfs -ls /user/training/sqoop_import/retail_db/order_itemsFound 5 items
-rw-r--r-- 1 cloudera supergroup 0 2019-11-02 03:43 /user/training/sqoop_import/retail_db/order_items/_SUCCESS
-rw-r--r-- 1 cloudera supergroup 363500 2019-11-02 03:43 /user/training/sqoop_import/retail_db/order_items/part-m-00000
-rw-r--r-- 1 cloudera supergroup 377289 2019-11-02 03:43 /user/training/sqoop_import/retail_db/order_items/part-m-00001
-rw-r--r-- 1 cloudera supergroup 389972 2019-11-02 03:43 /user/training/sqoop_import/retail_db/order_items/part-m-00002
-rw-r--r-- 1 cloudera supergroup 389759 2019-11-02 03:43 /user/training/sqoop_import/retail_db/order_items/part-m-00003
[cloudera@quickstart ~]$
No comments:
Post a Comment