In this talk, well walk you through the up-to-date status of Big Data on AArch64, especially focus on improvements weve made to support big data components(Hadoop, Spark, etc) across platform, the contribution to Apache Bigtop, and the future roadmap.
Apache Bigtop as the open source Hadoop distribution, provides full functionality of packaging, testing, and deployment for users to build their own Hadoop Distro from ground up. Because of its unique positioning, it is widely leveraged by not only ISVs(Cloudera, Hortonworks, Pivotal, ODPi, etc), but also cloud providers(AWE EMR, Google Cloud Dataproc). Laying on top of Bigtops foundation, weve pushed the support of big data to the next level under the support of Open Data Platform Initiative(ODPi). To breakdown the details of this talk, well share about the enhancement of AArch64 support in Bigtop, how to run big data on AArch64 with Bigtop Provisioner and Sandbox at a click of button, how to build the support of AArch64 with Bigtops docker based architecture, and how to test the result with a comprehensive CI coverage. Further more, the newly developed integration testing framework 2.0 in Apache Bigtop will be revealed.
ASF member, Apache Bigtop Committer/PMC member/Former VP, Director of Taiwan Data Engineering Association (Apache Software Foundation)
Yu-Hsin Yeh(Evans Ye) is former VP, and currently committer and PMC member of Apache Bigtop. He loves to code, automate things, and tackling big data challenges. Aside from engineering stuff, he is also an enthusiast in giving talks to share software innovations and cutting-edge technologies. His latest talk were to introduce Apache Calcite on DataCon.TW 2018. On the other hand, Evans had talked about Apache Bigtop’s several new features on Dataworks Summit 2017 San Jose, Apache: Big Data NA 2017/2016, EU 2015. He also presented the SDACK architecture on DockerCon 2016. In addition to that, 2 times Big Data Innovation Summit speaker, 2 times HadoopCon speaker, 2 times Taiwan Hadoop user group speaker, and dozens of company internal talks make him an experienced presenter.
Tech Lead (Linaro)
Sr. SW Engineering Manager (Arm)