November 13–15, 2018 - Shanghai, China
Click Here For Information & Registration

To view the Chinese version of this schedule please go here.

Simultaneous translation will be provided for all keynote and breakout sessions.
Back To Schedule
Thursday, November 15 • 16:00 - 16:35
Spark on Kubernetes: Best Practice and Performance - Junjie Chen & Jerry Shao, Tencent

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
As of version 2.3, Spark can run on clusters managed by Kubernetes, the de facto automation framework for contained based applications which is a significant milestone for k8s to support big data services. In this talk, firstly we will introduce our work for offering spark service via Kubernetes deployment as public cloud services, like: Authorization and Logging, and multi-tenancy through namespace and quota management of Kubernetes, etc. Then we will share the best practices of performance tuning details while running Spark application, includes: tuning detailed configurations from Kubernetes and Spark for maximum resource utilization, integrating with zookeeper service to achieve high availability, etc. In prospective of performance, the TPC-DS workload is used to present performance impact brought by configurations change.


Junjie Chen

Senior Software Engineer 高级软件工程师, Tencent
Chen, Junjie is working as Senior Software Engineer In Tencent Cloud. He has been working on big data and cloud area over several years and has rich experiences on SQL on Hadoop area. Currently, he is participating data warehouse project from Tencent Cloud. Chen Junjie... Read More →

Jerry Shao

Expert Engineer 专家工程师, Tencent 腾讯
Jerry Shao works as an expert engineer at Tencent Cloud, mainly focused on Spark area, especially Spark core, Spark on Yarn and Spark Streaming. He is an Apache Spark committer and Apache Livy PMC. Prior to Tencent, he was a Member of Technical Staff at Hortonworks working on open... Read More →

Thursday November 15, 2018 16:00 - 16:35 CST
2F Room 3
  • Skill Level Any