2025年Hive的安装及配置

Hive的安装及配置一 Hive 安装地址 1 Hive 官网地址 http hive apache org 2 文档查看地址 https cwiki apache org confluence display Hive GettingStart 3 下载地址 http archive apache org dist hive 二 Hive 安装部署 2 1

大家好,我是讯享网,很高兴认识大家。

一、Hive安装地址

1、Hive官网地址
http://hive.apache.org/

2、文档查看地址
https://cwiki.apache.org/confluence/display/Hive/GettingStarted

3、下载地址
http://archive.apache.org/dist/hive/

二、Hive安装部署

2.1 Hive安装及配置

1、把 apache-hive-1.2.1-bin.tar.gz 上传到 linux 的 /opt/software 目录下

[test@hadoop151 opt]$ cd software/ [test@hadoop151 software]$ ll 总用量  -rw-rw-r-- 1 test test  2月 14 09:48 apache-hive-1.2.1-bin.tar.gz -rw-rw-r--. 1 test test  1月 25 10:41 hadoop-2.7.2.tar.gz -rw-rw-r--. 1 test test  1月 25 09:33 jdk-8u144-linux-x64.tar.gz 

讯享网

2、解压 apache-hive-1.2.1-bin.tar.gz 到 /opt/module/ 目录下面

讯享网[test@hadoop151 software]$ tar -zxvf apache-hive-1.2.1-bin.tar.gz -C /opt/module/ 

3、修改 apache-hive-1.2.1-bin.tar.gz 的名称为 hive

[test@hadoop151 module]$ mv apache-hive-1.2.1-bin/ hive [test@hadoop151 module]$ ll 总用量 16 drwxr-xr-x. 15 test test 4096 2月 3 21:53 hadoop-2.7.2 drwxrwxr-x 8 test test 4096 2月 14 09:49 hive drwxrwxr-x 2 test test 4096 2月 4 19:16 input drwxr-xr-x. 8 test test 4096 7月 22 2017 jdk1.8.0_144 

4、修改/opt/module/hive/conf 目录下的 hive-env.sh.template 名称为 hive-env.sh

讯享网[test@hadoop151 conf]$ mv hive-env.sh.template hive-env.sh 

5、配置 hive-env.sh 文件
(1) 配置 HADOOP_HOME 路径

export HADOOP_HOME=/opt/module/hadoop-2.7.2 

(2) 配置 HIVE_CONF_DIR 路径

讯享网export HIVE_CONF_DIR=/opt/module/hive/conf 

2.2 Hadoop集群配置

1、必须启动 hdfs 和 yarn

[test@hadoop151 conf]$ start-dfs.sh Starting namenodes on [hadoop151] hadoop151: starting namenode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-test-namenode-hadoop151.out hadoop151: starting datanode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-test-datanode-hadoop151.out hadoop152: starting datanode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-test-datanode-hadoop152.out hadoop153: starting datanode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-test-datanode-hadoop153.out Starting secondary namenodes [hadoop153] hadoop153: starting secondarynamenode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-test-secondarynamenode-hadoop153.out 
讯享网[test@hadoop152 ~]$ start-yarn.sh starting yarn daemons starting resourcemanager, logging to /opt/module/hadoop-2.7.2/logs/yarn-test-resourcemanager-hadoop152.out hadoop153: starting nodemanager, logging to /opt/module/hadoop-2.7.2/logs/yarn-test-nodemanager-hadoop153.out hadoop151: starting nodemanager, logging to /opt/module/hadoop-2.7.2/logs/yarn-test-nodemanager-hadoop151.out hadoop152: starting nodemanager, logging to /opt/module/hadoop-2.7.2/logs/yarn-test-nodemanager-hadoop152.out 

2、在 HDFS 上创建 /tmp 和 /user/hive/warehouse 两个目录并修改他们的同组权限可写(可不操作,系统会自动创建)

[test@hadoop153 ~]$ hadoop fs -mkdir /tmp [test@hadoop153 ~]$ hadoop fs -mkdir -p /user/hive/warehouse 
讯享网[test@hadoop153 ~]$ hadoop fs -chmod g+w /tmp [test@hadoop153 ~]$ hadoop fs -chmod g+w /user/hive/warehouse 

2.3 Hive基本操作

1、启动hive

[test@hadoop151 bin]$ hive Logging initialized using configuration in jar:file:/opt/module/hive/lib/hive-common-1.2.1.jar!/hive-log4j.properties hive> 

2、查看数据库


讯享网

讯享网hive> show databases; OK default Time taken: 1.355 seconds, Fetched: 1 row(s) 

3、打开默认的数据库

hive> use default; OK Time taken: 0.075 seconds 

4、显示 default 数据库中的表

讯享网hive> show tables; OK Time taken: 0.063 seconds 

5、创建一张表

hive> create table student(id int, name string); OK Time taken: 0.584 seconds 

6、显示数据库中有几张表

讯享网hive> show tables; OK student Time taken: 0.022 seconds, Fetched: 1 row(s) 

7、查看表的结构

hive> desc student; OK id int name string Time taken: 0.206 seconds, Fetched: 2 row(s) 

8、向表中插入数据

讯享网hive> insert into student values(1000,"ss"); Query ID = test_152_6f97830a-f3f8-4c26-9744-c77441cefd05 Total jobs = 3 Launching Job 1 out of 3 Number of reduce tasks is set to 0 since there's no reduce operator Starting Job = job_24_0001, Tracking URL = http://hadoop152:8088/proxy/application_24_0001/ Kill Command = /opt/module/hadoop-2.7.2/bin/hadoop job -kill job_24_0001 Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0 2020-02-14 10:42:18,853 Stage-1 map = 0%, reduce = 0% 2020-02-14 10:42:38,866 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.2 sec MapReduce Total cumulative CPU time: 4 seconds 200 msec Ended Job = job_24_0001 Stage-4 is selected by condition resolver. Stage-3 is filtered out by condition resolver. Stage-5 is filtered out by condition resolver. Moving data to: hdfs://hadoop151:9000/user/hive/warehouse/student/.hive-staging_hive_2020-02-14_10-41-52_166_0-1/-ext-10000 Loading data to table default.student Table default.student stats: [numFiles=1, numRows=1, totalSize=8, rawDataSize=7] MapReduce Jobs Launched: Stage-Stage-1: Map: 1 Cumulative CPU: 4.2 sec HDFS Read: 3557 HDFS Write: 79 SUCCESS Total MapReduce CPU Time Spent: 4 seconds 200 msec OK Time taken: 49.347 seconds 

9、查询表中数据

hive> select * from student; OK 1000 ss Time taken: 0.098 seconds, Fetched: 1 row(s) 

10、退出hive

讯享网hive> quit; 

三、使用MySQL存储Metastore

3.1 为何使用MySQL储存Metastore?

如果我们已经打开了一个 hive 窗口,那么当我们再打开一个客户端启动 hive,会产生 java.sql.SQLException 异常

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient 

原因是:Metastore 默认存储在自带的 derby 数据库中,推荐使用 MySQL 存储 Metastore。

3.2 MySQL安装

1、查看 mysql 是否安装,如果安装了,卸载 mysql

(1) 查看

讯享网[test@hadoop151 ~]$ rpm -qa | grep mysql mysql-libs-5.1.73-7.e
小讯
上一篇 2025-02-06 17:58
下一篇 2025-03-08 10:03

相关推荐

版权声明:本文内容由互联网用户自发贡献,该文观点仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容,请联系我们,一经查实,本站将立刻删除。
如需转载请保留出处:https://51itzy.com/kjqy/78474.html