Loading…
November 13–15, 2018 - Shanghai, China
Click Here For Information & Registration

To view the Chinese version of this schedule please go here.
请点击此处查看中文版本。

我们将为所有主题演讲和分组会议提供同声传译服务。
Simultaneous translation will be provided for all keynote and breakout sessions.
Thursday, November 15 • 16:00 - 16:35
Spark on Kubernetes: Best Practice and Performance - Junjie Chen & Jerry Shao, Tencent

Sign up or log in to save this to your schedule and see who's attending!

Feedback form is now closed.
As of version 2.3, Spark can run on clusters managed by Kubernetes, the de facto automation framework for contained based applications which is a significant milestone for k8s to support big data services. In this talk, firstly we will introduce our work for offering spark service via Kubernetes deployment as public cloud services, like: Authorization and Logging, and multi-tenancy through namespace and quota management of Kubernetes, etc. Then we will share the best practices of performance tuning details while running Spark application, includes: tuning detailed configurations from Kubernetes and Spark for maximum resource utilization, integrating with zookeeper service to achieve high availability, etc. In prospective of performance, the TPC-DS workload is used to present performance impact brought by configurations change.

Speakers
JC

Junjie Chen

Senior Software Engineer 高级软件工程师, Tencent
Chen, Junjie is working as Senior Software Engineer In Tencent Cloud. He has been working on big data and cloud area over several years and has rich experiences on SQL on Hadoop area. Currently, he is participating data warehouse project from Tencent Cloud. Chen Junjie... Read More →
JS

Jerry Shao

Expert Engineer 专家工程师, Tencent 腾讯
Jerry Shao works as an expert engineer at Tencent Cloud, mainly focused on Spark area, especially Spark core, Spark on Yarn and Spark Streaming. He is an Apache Spark committer and Apache Livy PMC. Prior to Tencent, he was a Member of Technical Staff at Hortonworks working on open... Read More →



Thursday November 15, 2018 16:00 - 16:35
2F Room 3
  • Skill Level Any