Bigtop

ASF Gitbox:

https://gitbox.apache.org/repos/asf/bigtop.git

Aarch64 & Bigtop

• https://sched.co/LJq5
• https://www.youtube.com/watch?v=a0acq7zI1wI

bigpetstore:

• https://www.slideshare.net/MrtonBalassi/implementing-bigpetstore-with-apache-flink

Use case:

• Integration Test: https://www.slideshare.net/Hadoop_Summit/scaling-hadoop-at-linkedin-107176757

Ambari https://issues.apache.org/jira/browse/AMBARI-19429

https://dataworkssummit.com/san-jose-2017/sessions/leveraging-docker-for-hadoop-build-automation-and-big-data-stack-provisioning/

http://events.linuxfoundation.org/sites/events/files/slides/Leveraging%20Docker%20for%20Hadoop%20Build%20Automation%20and%20Big%20Data%20Stack%20Provisioning.pdf

Docs @ Linaro

• https://collaborate.linaro.org/display/BDTS/Documentation

Docker & CI:

• https://issues.apache.org/jira/browse/BIGTOP-2811

patch:

git format-patch HEAD^..HEAD --stdout > BIGTOP-1031.patch

apply the patch:

curl https://patch-diff.githubusercontent.com/raw/apache/bigtop/pull/442.patch | git am --signoff

docker run -ti --rm --privileged -v `pwd`:/bigtop bigtop/slaves:trunk-centos-7 bash -l

docker run -ti --rm -v `pwd`:/bigtop bigtop/slaves:trunk-ubuntu-16.04 bash -l

-- user namesapce
docker run -ti --rm -u jenkins -v `pwd`:/bigtop bigtop/slaves:trunk-centos-7 bash -l

CentOS 7:

yum update -y java\*
keytool -import -alias hynix -keystore  /usr/lib/jvm/java-1.8.0/jre/lib/security/cacerts -file /bigtop/sslproxy.crt

With R support:

docker run -ti -u jenkins:jenkins --rm -v `pwd`:/bigtop youngwookim/bigtop-slaves:centos-7 bash -l

BOM:

./gradlew -Dbomfile=metatron.bom zeppelin-clean zeppelin-rpm

http://blogs.aws.amazon.com/bigdata/post/TxNJ6YS4X6S59U/Building-and-Deploying-Custom-Applications-with-Apache-Bigtop-and-Amazon-EMR

Ansible roles and playbooks for Bigtop, https://issues.apache.org/jira/browse/BIGTOP-1584

package tests are for testing the correctness of the packages through the development cycle. They are usually pretty trivial and you can find the examples of those under bigtop-tests/test-artifacts/package/

My bigtop-slaves:

FROM bigtop/slaves:trunk-centos-7

RUN yum install -y epel-release
RUN yum install -y R R-devel libcurl-devel openssl-devel
RUN yum clean all

docker build --no-cache --force-rm -t youngwookim/bigtop-slaves:centos-7 .
docker push youngwookim/bigtop-slaves:centos-7

https://registry.yarnpkg.com/grunt-angular-templates: unable to verify the first certificate

~/.yarnrc:
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1
strict-ssl false

Metatron DAP:

• YUM

http://52.79.212.190/yum/metatron/centos-6/
http://52.79.212.190/yum/metatron/centos-7/

docker run -ti -u jenkins:jenkins --rm -v `pwd`:/bigtop youngwookim/bigtop-slaves:centos-6 bash -l
docker run -ti -u jenkins:jenkins --rm -v `pwd`:/bigtop youngwookim/bigtop-slaves:centos-7 bash -l

cd /bigtop

./gradlew -Dbomfile=metatron.bom allclean ; ./gradlew -Dbomfile=metatron.bom spark-rpm zeppelin-rpm tez-rpm bigtop-groovy-rpm bigtop-jsvc-rpm bigtop-tomcat-rpm bigtop-utils-rpm zookeeper-rpm hadoop-rpm hive-rpm hbase-rpm phoenix-rpm hue-rpm flume-rpm kafka-rpm flink-rpm presto-rpm ; ./gradlew -Dbomfile=metatron.bom yum

https://insights.ubuntu.com/2016/06/07/announcing-charms-for-apache-bigtop-big-data/