一、Hive安装地址
1、Hive官网地址
http://hive.apache.org/
2、文档查看地址
https://cwiki.apache.org/confluence/display/Hive/GettingStarted
3、下载地址
http://archive.apache.org/dist/hive/
二、Hive安装部署
2.1 Hive安装及配置
1、把 apache-hive-1.2.1-bin.tar.gz 上传到 linux 的 /opt/software 目录下
[test@hadoop151 opt]$ cd software/ [test@hadoop151 software]$ ll 总用量 -rw-rw-r-- 1 test test 2月 14 09:48 apache-hive-1.2.1-bin.tar.gz -rw-rw-r--. 1 test test 1月 25 10:41 hadoop-2.7.2.tar.gz -rw-rw-r--. 1 test test 1月 25 09:33 jdk-8u144-linux-x64.tar.gz
讯享网
2、解压 apache-hive-1.2.1-bin.tar.gz 到 /opt/module/ 目录下面
讯享网[test@hadoop151 software]$ tar -zxvf apache-hive-1.2.1-bin.tar.gz -C /opt/module/
3、修改 apache-hive-1.2.1-bin.tar.gz 的名称为 hive
[test@hadoop151 module]$ mv apache-hive-1.2.1-bin/ hive [test@hadoop151 module]$ ll 总用量 16 drwxr-xr-x. 15 test test 4096 2月 3 21:53 hadoop-2.7.2 drwxrwxr-x 8 test test 4096 2月 14 09:49 hive drwxrwxr-x 2 test test 4096 2月 4 19:16 input drwxr-xr-x. 8 test test 4096 7月 22 2017 jdk1.8.0_144
4、修改/opt/module/hive/conf 目录下的 hive-env.sh.template 名称为 hive-env.sh
讯享网[test@hadoop151 conf]$ mv hive-env.sh.template hive-env.sh
5、配置 hive-env.sh 文件
(1) 配置 HADOOP_HOME 路径
export HADOOP_HOME=/opt/module/hadoop-2.7.2
(2) 配置 HIVE_CONF_DIR 路径
讯享网export HIVE_CONF_DIR=/opt/module/hive/conf
2.2 Hadoop集群配置
1、必须启动 hdfs 和 yarn
[test@hadoop151 conf]$ start-dfs.sh Starting namenodes on [hadoop151] hadoop151: starting namenode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-test-namenode-hadoop151.out hadoop151: starting datanode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-test-datanode-hadoop151.out hadoop152: starting datanode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-test-datanode-hadoop152.out hadoop153: starting datanode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-test-datanode-hadoop153.out Starting secondary namenodes [hadoop153] hadoop153: starting secondarynamenode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-test-secondarynamenode-hadoop153.out
讯享网[test@hadoop152 ~]$ start-yarn.sh starting yarn daemons starting resourcemanager, logging to /opt/module/hadoop-2.7.2/logs/yarn-test-resourcemanager-hadoop152.out hadoop153: starting nodemanager, logging to /opt/module/hadoop-2.7.2/logs/yarn-test-nodemanager-hadoop153.out hadoop151: starting nodemanager, logging to /opt/module/hadoop-2.7.2/logs/yarn-test-nodemanager-hadoop151.out hadoop152: starting nodemanager, logging to /opt/module/hadoop-2.7.2/logs/yarn-test-nodemanager-hadoop152.out
2、在 HDFS 上创建 /tmp 和 /user/hive/warehouse 两个目录并修改他们的同组权限可写(可不操作,系统会自动创建)
[test@hadoop153 ~]$ hadoop fs -mkdir /tmp [test@hadoop153 ~]$ hadoop fs -mkdir -p /user/hive/warehouse
讯享网[test@hadoop153 ~]$ hadoop fs -chmod g+w /tmp [test@hadoop153 ~]$ hadoop fs -chmod g+w /user/hive/warehouse
2.3 Hive基本操作
1、启动hive
[test@hadoop151 bin]$ hive Logging initialized using configuration in jar:file:/opt/module/hive/lib/hive-common-1.2.1.jar!/hive-log4j.properties hive>
2、查看数据库
讯享网hive> show databases; OK default Time taken: 1.355 seconds, Fetched: 1 row(s)
3、打开默认的数据库
hive> use default; OK Time taken: 0.075 seconds
4、显示 default 数据库中的表
讯享网hive> show tables; OK Time taken: 0.063 seconds
5、创建一张表
hive> create table student(id int, name string); OK Time taken: 0.584 seconds
6、显示数据库中有几张表
讯享网hive> show tables; OK student Time taken: 0.022 seconds, Fetched: 1 row(s)
7、查看表的结构
hive> desc student; OK id int name string Time taken: 0.206 seconds, Fetched: 2 row(s)
8、向表中插入数据
讯享网hive> insert into student values(1000,"ss"); Query ID = test_152_6f97830a-f3f8-4c26-9744-c77441cefd05 Total jobs = 3 Launching Job 1 out of 3 Number of reduce tasks is set to 0 since there's no reduce operator Starting Job = job_24_0001, Tracking URL = http://hadoop152:8088/proxy/application_24_0001/ Kill Command = /opt/module/hadoop-2.7.2/bin/hadoop job -kill job_24_0001 Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0 2020-02-14 10:42:18,853 Stage-1 map = 0%, reduce = 0% 2020-02-14 10:42:38,866 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.2 sec MapReduce Total cumulative CPU time: 4 seconds 200 msec Ended Job = job_24_0001 Stage-4 is selected by condition resolver. Stage-3 is filtered out by condition resolver. Stage-5 is filtered out by condition resolver. Moving data to: hdfs://hadoop151:9000/user/hive/warehouse/student/.hive-staging_hive_2020-02-14_10-41-52_166_0-1/-ext-10000 Loading data to table default.student Table default.student stats: [numFiles=1, numRows=1, totalSize=8, rawDataSize=7] MapReduce Jobs Launched: Stage-Stage-1: Map: 1 Cumulative CPU: 4.2 sec HDFS Read: 3557 HDFS Write: 79 SUCCESS Total MapReduce CPU Time Spent: 4 seconds 200 msec OK Time taken: 49.347 seconds
9、查询表中数据
hive> select * from student; OK 1000 ss Time taken: 0.098 seconds, Fetched: 1 row(s)
10、退出hive
讯享网hive> quit;
三、使用MySQL存储Metastore
3.1 为何使用MySQL储存Metastore?
如果我们已经打开了一个 hive 窗口,那么当我们再打开一个客户端启动 hive,会产生 java.sql.SQLException 异常
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
原因是:Metastore 默认存储在自带的 derby 数据库中,推荐使用 MySQL 存储 Metastore。
3.2 MySQL安装
1、查看 mysql 是否安装,如果安装了,卸载 mysql
(1) 查看
讯享网[test@hadoop151 ~]$ rpm -qa | grep mysql mysql-libs-5.1.73-7.e

版权声明:本文内容由互联网用户自发贡献,该文观点仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容,请联系我们,一经查实,本站将立刻删除。
如需转载请保留出处:https://51itzy.com/kjqy/78474.html