Apache Spark & Scala Content
Introduction to Big Data
What is Big Data
Challenges with Big Data
Batch Vs. Real Time Big Data Analytics
Batch Analytics
- Hadoop Ecosystem Overview
Real Time Analytics
Streaming Data - Storm
In Memory Data - Spark
Introduction of Spark
What is Spark
Why Spark
Who Uses Spark
Brief History of Spark
Storage Layers for Spark
Unified Stack of Spark
- Spark Core
- Spark Sql
- Spark Streaming
- MLib
- GraphX
Modes of Spark Installation
Learn Spark more ..
Installation Spark in different modes
- Local mode
- Pseudo mode
- Cluster mode
Spark Architecture explanation
- Master Slave architecture
- Spark Driver
- Workers
- Executors
Working with Spark in different programming languages
Python
- How to use 'pyspark'
- Practical examples on spark in python
Scala
- How to use 'spark-shell'
- Practical examples on spark in Scala
Java
- How to use 'Java'
- Practical examples on spark in Java
R
- How to use 'SparkR'
- Practical examples on spark in R
Creating the Spark Context
Configuring Spark Context with SparkConf
Caching Overview
Distributed Persistence
Combine scala and java seamlessly
Deploying Applications with spark-submit
Verify spark jobs in Spark Web UI
Installing sbt
Building a Spark Project with sbt
Running Spark Project with sbt
Installing maven
Building a Spark Project with maven
Running Spark Project with maven
Resilient Distributed Dataset (RDD)
What is RDD
Creating RDDs
RDD Operations
- Transformations
- Actions
- Lazy Evaluation
Passing Functions to Spark
Working with Key/Value Pairs
Creating Pair RDDs
Transformations on Pair RDDs
- Aggregations
- Grouping Data
- Joins
- Sorting Data
Data Partitioning
- Determining an RDD’s Partitioner
- Custom Partitioners
Loading and Saving Your Data
File Formats
- Text, Json, csv, tsv, Object files
- Hadoop Input and Output Formats
Loading Data using RDD
Saving Data using RDD
MapReduce and Pair RDD Operations
Scala and Hadoop Integrations
Broadcast and Accumulators
Accumulators
- Introduction to Accumulators
- Practical Examples on Accumulators
- Creating Custom Accumulators
Broadcast variables
- Introduction to Broadcast variables
- Practical Examples on Broadcast variables
Apache Spark SQL
Hive and Spark SQL Architecture explanation
Working with Spark SQL DataFrames
Using Spark SQL Context
Practical examples on Spark SQL
Integrating hive and Spark SQL
Creating & Using SQL Hive Context
Hive Queries through Spark
Processing Text, JSON and Parquet Files using in Spark
Spark SQL UDFs
Spark SQL Performance Tuning Options
JDBC/ODBC Server
Apache Spark Streaming
Spark Streaming Architecture explanation
Transformations
Output Operations
Streaming UI explanation
Performance Considerations
Practical examples on Spark Streaming
Apache Spark MLib
Machine Learning Basics
Machine Learning Algorithms
- Classification
- Clustering
- Collaborative Filtering
Performance Considerations
Practical examples on Spark MLib
Apache Mesos
Introduction to Apache Mesos
Apache Mesos Architecture explanation
Practical Examples on Apache Mesos
Apache Mahout
Introduction to Apache Mahout
Apache Mahout Architecture explanation
Practical Examples on Apache Mahout
Apache Storm
Introduction to Apache Storm
Apache Storm Architecture explanation
Practical Examples on Apache Storm
Apache Kafka
Introduction to Apache Kafka
Apache Kafka Architecture explanation
Practical Examples on Apache Kafka
Introduction of Scala
What is Scala?
Why Scala?
Advantages of Scala?
Using the Scala REPL(Read Evaluate print loop)
What is Type Inference
Interoperability between Scala and Java
Scala using Command Line
Installing Java & Scala
Interactive Scala
Writing Scala Scripts
Compiling Scala Programs
Basics of Scala
Defining Variables
Defining Functions
String Interpolation
IDE for Scala
Scala Type Less, Do More
Semicolons
Variable Declarations
Method Declarations
Type Inference
Immutability
Reserved Words
Operators
Precedence Rules
Literals
Options
Arrays, Lists, Ranges, Tuples
Expressions and Conditionals
If expressions
If-Else expressions
Match Expressions
For Loops
While Loops
Do-While Loops
Conditional Operators
Enumerations
Pattern Matching
Using try, catch, and finally Clauses
Functional Programming in Scala
What is Functional Programming?
Functional Literals and Closures
Recursions
Tail Calls
Currying
Functional Data Structures
Traversing
- Traversal, Mapping, Filtering, Folding and Reducing
Implicit Function Parameters
Call by Name, Call by Value
Object-Oriented Programming in Scala
Class and Object Basics
Value Classes
Parent Types
Constructors in Scala
Fields in Classes
Nested Types
Traits as Mixins
Stackable Traits
Creating Traits
Visibility Rules
Scala for Big Data
Improving MapReduce with Scala
Moving Beyond MapReduce
Categories for Mathematics
A List of Scala-Based Data Tools
What we are offering to you:
- Hands on Practice on Spark & Scala Real-Time Examples.
- Providing 1 Major project on Spark.
- Providing 2 Mini projects on Spark.
- Hands on installation Spark and it's relative software's in your laptop.
- Well documented Spark & Scala material with all the topics covering in the course.
- Well documented Spark blog contains frequent interview questions along with the answers and latest updates on BigData technology.
- Discussing about Spark & Scala interview questions daily base.
- Resume preparation with POC's or Project's based on your experience.
