杭州Cloudera Apache Spark程序员

Cloudera Apache Spark程序员 培训班型： 公开课，内训 课程长度： 3天/18小时 培训日期： 待定 认证考试： 暂无 培训地点： 博学国际教育培训中心 环境要求： 投影仪、白板、大白纸 培训形式： 实例讲授，现场演、练、及时沟通 培训资料： 培训教材 课程内容 Cloudera Developer Training for Apache Spark 课程概述： 结合批处理、流媒体和交互分析技术，利用 Apache Spark 构建完整统一的大 数据应用。学习编写复杂的并行应用程序，为各种用例、架构和行业执行快速良好的决策和实时行动。 授课对象： 面向意欲优化应用程序速度、易用性和复杂程度的开发人员和工程师。培训对象要求 具 备Python或Scala背景知识，具备Linux 相关基础知识更佳。 培训目标： Using the Spark shell for interactive data analysis  The features of Spark’s Resilient Distributed Datasets  How Spark runs on a cluster  How Spark parallelizes task execution  Writing Spark applications  Processing streaming data with Spark 课程内容： Introduction to Spark  What is Spark?  Review: From Hadoop MapReduce to Spark  Review: HDFS  Review: YARN  Spark Overview Spark Basics  Using the Spark Shell  RDDs (Resilient Distributed Datasets)  Functional Programming in Spark Working with RDDs in Spark  Creating RDDs  Other General RDD Operations Aggregating Data with Pair RDDs  Key-Value Pair RDDs  Map-Reduce  Other Pair RDD Operations Writing and Deploying Spark Applications  Spark Applications vs. Spark Shell  Creating the SparkContext  Building a Spark Application (Scala and Java)  Running a Spark Application  The Spark Application Web UI  Hands-On Exercise: Write and Run a Spark Application  Configuring Spark Properties  Logging Parallel Processing  Review: Spark on a Cluster  RDD Partitions  Partitioning of File-based RDDs  HDFS and Data Locality  Executing Parallel Operations  Stages and Tasks Spark RDD Persistence  RDD Lineage  RDD Persistence Overview  Distributed Persistence Basic Spark Streaming  Spark Streaming Overview  Example: Streaming Request Count  DStreams  Developing Spark Streaming Applications Advanced Spark Streaming  Multi-Batch Operations  State Operations  Sliding Window Operations  Advanced Data Sources Common Patterns in Spark Data Processing  Common Spark Use Cases  Iterative Algorithms in Spark  Graph Processing and Analysis  Machine Learning  Example: k-means Improving Spark Performance  Shared Variables: Broadcast Variables  Shared Variables: Accumulators  Common Performance Issues  Diagnosing Performance Problems Spark SQL and DataFrames  Spark SQL and the SQL Context  Creating DataFrames  Transforming and Querying DataFrames  Saving DataFrames  DataFrames and RDDs  Comparing Spark SQL, Impala and Hive-on-Spark