怎么编译Hadoop2.7.6
本篇内容介绍了“怎么编译Hadoop2.7.6”的有关知识,在实际案例的操作过程中,不少人都会遇到这样的困境,接下来就让小编带领大家学习一下如何处理这些情况吧!希望大家仔细阅读,能够学有所成!
成都创新互联成立10年来,这条路我们正越走越好,积累了技术与客户资源,形成了良好的口碑。为客户提供成都网站建设、成都网站制作、网站策划、网页设计、空间域名、网络营销、VI设计、网站改版、漏洞修补等服务。网站是否美观、功能强大、用户体验好、性价比高、打开快等等,这些对于网站建设都非常重要,成都创新互联通过对建站技术性的掌握、对创意设计的研究为客户提供一站式互联网解决方案,携手广大客户,共同发展进步。
环境:虚拟机
脚本
#!/usr/bin/env bash set -e # 软件准备 # JDK下载地址 https://www.oracle.com/java/technologies/oracle-java-archive-downloads.html # 域名 admin.xdata.dongjihui.com 当前解析内网地址,公网无法访问,请自行下载jdk。 wget -cP /data/software/oracle/jdk/ http://admin.xdata.dongjihui.com/data/software/oracle/jdk/jdk-8u202-linux-x64.tar.gz wget -cP /data/software/apache/hadoop/ https://archive.apache.org/dist/hadoop/common/hadoop-2.7.6/hadoop-2.7.6.tar.gz wget -cP /data/software/apache/hadoop/ https://archive.apache.org/dist/hadoop/common/hadoop-2.7.6/hadoop-2.7.6-src.tar.gz wget -cP /data/software/apache/maven/ https://mirrors.tuna.tsinghua.edu.cn/apache/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz wget -cP /data/software/apache/ant/ https://mirrors.tuna.tsinghua.edu.cn/apache/ant/binaries/apache-ant-1.9.15-bin.tar.gz wget -cP /data/software/google/protobuf/ https://github.com/google/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.gz wget -cP /data/software/google/snappy/ https://github.com/google/snappy/archive/1.1.7.tar.gz # 依赖库 sudo apt-get install -y gcc g++ subversion libapache2-mod-svn git wget curl build-essential autoconf automake libtool cmake zlib1g-dev libbz2-dev pkg-config libssl1.0-dev # 创建目录 mkdir -p /data/app/java/ /data/app/apache/hadoop /data/app/apache/ant /data/app/apache/maven /data/src # 安装protobuf cd /data/src tar zxf /data/software/google/protobuf/protobuf-2.5.0.tar.gz cd protobuf-2.5.0/ ./configure make make check sudo make install sudo ldconfig # 安装snappy cd /data/src/ tar zxf /data/software/google/snappy/1.1.7.tar.gz cd snappy-1.1.7/ mkdir build && cd build && cmake ../ && make sudo make install # 安装jdk cd /data/app/java/ tar zxf /data/software/oracle/jdk/jdk-8u202-linux-x64.tar.gz ln -s jdk1.8.0_202 cur_jdk # 安装ant cd /data/app/apache/ant/ tar zxf /data/software/apache/ant/apache-ant-1.9.15-bin.tar.gz ln -s apache-ant-1.9.15 cur_ant # 安装并配置maven cd /data/app/apache/maven tar zxf /data/software/apache/maven/apache-maven-3.5.4-bin.tar.gz ln -s apache-maven-3.5.4 cur_maven sed \ -e 54a'\/data/app/apache/maven/repo\ ' \ -e 158a'\ \' \ -e 158a'\ \ ' \ -i cur_maven/conf/settings.xml # 解压hadoop,用于验证本地库 cd /data/app/apache/hadoop tar zxf /data/software/apache/hadoop/hadoop-2.7.6.tar.gz ln -s hadoop-2.7.6 cur_hadoop # 配置环境变量 cat>env4app.sh<<-'EOF' export JAVA_HOME=/data/app/java/cur_jdk export CLASSPATH=.:${JAVA_HOME}/lib:${JAVA_HOME}/jre/lib:${CLASSPATH} export M2_HOME=/data/app/apache/maven/cur_maven export ANT_HOME=/data/app/apache/ant/cur_ant export HADOOP_PREFIX=/data/app/apache/hadoop/cur_hadoop export TMPPATH=.:${JAVA_HOME}/bin:${M2_HOME}/bin:${ANT_HOME}/bin:${HADOOP_PREFIX}/bin export PATH=${TMPPATH}:$PATH EOF if [ -f /etc/profile.d/env4app.sh ]; then sudo mv /etc/profile.d/env4app.sh{,.bak-`date +%Y%m%d%H%M%S`}; fi sudo mv env4app.sh /etc/profile.d/; source /etc/profile source ~/.profile # 编译 cd /data/src/ tar zxf /data/software/apache/hadoop/hadoop-2.7.6-src.tar.gz cd hadoop-2.7.6-src/ mvn clean package -Pdist,native -DskipTests -Dtar # 部署 cd ${HADOOP_PREFIX}/lib/ mv native{,.bak} cp -r /data/src/hadoop-2.7.6-src/hadoop-dist/target/hadoop-2.7.6/lib/native . cd ${HADOOP_PREFIX}/ # 检查 bin/hadoop checknative -a echo "执行完毕@`date +%c`"aliyunmaven\ ' \ -e 158a'\ \*\ ' \ -e 158a'\ \阿里云公共仓库\ ' \ -e 158a'\ \https://maven.aliyun.com/repository/public\ ' \ -e 158a'\ \
“怎么编译Hadoop2.7.6”的内容就介绍到这里了,感谢大家的阅读。如果想了解更多行业相关的知识可以关注创新互联网站,小编将为大家输出更多高质量的实用文章!
文章标题:怎么编译Hadoop2.7.6
文章来源:http://scgulin.cn/article/iedscg.html