且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

如何在不使用Sqoop的情况下自动将Hive外部表与MySQL表同步?

更新时间:2022-03-26 22:19:55

如果没有任何发现,您可以创建表STORED BY JdbcStorageHandler .项目存储库: https://github.com/qubole/Hive-JDBC- Storage-Handler 它将像通常的蜂巢表一样工作,但是查询将在MySQL上运行.谓词下推将起作用.

Without scoop, you can create table STORED BY JdbcStorageHandler. Project repository: https://github.com/qubole/Hive-JDBC-Storage-Handler It will work as usual hive table, but query will run on MySQL. Predicate pushdown will work.

DROP TABLE HiveTable;
CREATE EXTERNAL TABLE HiveTable(
  id INT,
  id_double DOUBLE,
  names STRING,
  test INT
)
STORED BY 'org.apache.hadoop.hive.jdbc.storagehandler.JdbcStorageHandler'
TBLPROPERTIES (
  "mapred.jdbc.driver.class"="com.mysql.jdbc.Driver",
  "mapred.jdbc.url"="jdbc:mysql://localhost:3306/rstore",
  "mapred.jdbc.username"="root",
  "mapred.jdbc.input.table.name"="JDBCTable",
  "mapred.jdbc.output.table.name"="JDBCTable",
  "mapred.jdbc.password"="",
  "mapred.jdbc.hive.lazy.split"= "false"
);