Module 1: Introduction to Big Data and Hadoop Ecosystem

·       Understanding the "3 Vs" of Big Data

·       The evolution of big data technology and challenges

·       Overview of the Hadoop ecosystem components

·       Big Data career paths and industry trends

·       Practical session: Setting up a pseudo-distributed Hadoop environment

Module 2: Hadoop Distributed File System (HDFS)

·       HDFS architecture and its components (NameNode, DataNode)

·       HDFS commands for file and directory operations

·       Understanding data replication and fault tolerance

·       Advanced concepts like federation and snapshots

·       Practical session: Performing file operations on a distributed file system using HDFS shell commands

Module 3: Hadoop MapReduce Framework

·       MapReduce architecture and data flow

·       Writing and executing a simple MapReduce program

·       Understanding Mappers, Reducers, and Combiners

·       Debugging and monitoring MapReduce jobs

·       Practical session: Writing and running a MapReduce application to count words in a large text file

Module 4: Introduction to Apache Spark

·       Spark vs. Hadoop MapReduce: A comparative analysis

·       Spark's architecture and its key components (Driver, Executor, Cluster Manager)

·       The role of in-memory computing and its benefits

·       An overview of Spark's APIs and languages (Scala, Python, Java)

·       Practical session: Running a basic Spark application on the command line

Module 5: Spark Core and Resilient Distributed Datasets (RDDs)

·       Introduction to RDDs: their characteristics and advantages

·       Transformations vs. Actions and their lazy evaluation

·       Caching and persistence of RDDs for performance

·       Partitioning and data distribution strategies

·       Practical session: Using RDD transformations to filter and process a large dataset

Module 6: Spark SQL and DataFrames

·       Spark SQL architecture and its components

·       Introduction to DataFrames and their schema-based structure

·       Working with structured and semi-structured data

·       Writing and executing SQL queries on DataFrames

·       Practical session: Loading data from a JSON file into a DataFrame and performing SQL queries on it

Module 7: Spark Structured Streaming

·       Understanding the concept of real-time data processing

·       Introduction to Structured Streaming and its continuous processing model

·       Building a streaming pipeline from various sources (e.g., files, sockets)

·       Stateful vs. Stateless operations in streaming

·       Practical session: Creating a real-time word count application that processes data from a streaming source

Module 8: Spark Machine Learning Library (MLlib)

·       Overview of MLlib and its key features

·       Introduction to machine learning pipelines and the Pipeline API

·       Implementing classification and clustering algorithms

·       Model evaluation and hyperparameter tuning

·       Practical session: Building a simple linear regression model using MLlib on a sample dataset

Module 9: Graph Analytics with Spark GraphX

·       Introduction to graph processing and its applications

·       GraphX API: Vertices, Edges, and Properties

·       Common graph algorithms (e.g., PageRank, Connected Components)

·       Building and manipulating graphs

·       Practical session: Applying the PageRank algorithm to a web graph dataset

Module 10: Real-world Big Data Architectures

·       Lambda and Kappa architectures

·       Designing end-to-end data pipelines

·       Choosing the right tools for each stage of the pipeline

·       Data governance and lineage in a big data ecosystem

·       Practical session: Designing a conceptual architecture for a real-time e-commerce analytics system

Module 11: Data Ingestion with Apache Sqoop and Flume

·       Introduction to data ingestion and its challenges

·       Using Sqoop to import data from relational databases to HDFS

·       Using Flume to ingest streaming data from various sources

·       Best practices for batch and streaming data ingestion

·       Practical session: Ingesting data from a MySQL database into HDFS using Sqoop

Module 12: Messaging and Streaming with Apache Kafka

·       Introduction to Kafka: Topics, Producers, and Consumers

·       Building a reliable and scalable messaging system

·       Integrating Kafka with Spark Streaming

·       Advanced Kafka concepts like partitioning and replication

·       Practical session: Setting up a Kafka cluster and producing/consuming messages

Module 13: Data Warehousing with Hive and Impala

·       Introduction to Hive and its architecture

·       Writing HiveQL queries for data analysis

·       Understanding the role of Impala for low-latency queries

·       Comparing Hive and Impala and their use cases

·       Practical session: Creating Hive tables and running complex queries for a data warehousing scenario

Module 14: NoSQL Databases in the Big Data Landscape

·       Understanding different NoSQL database types (key-value, document, column-family)

·       Introduction to HBase and its integration with Hadoop

·       Choosing the right NoSQL database for your use case

·       Data modeling for NoSQL databases

·       Practical session: Interacting with an HBase database to store and retrieve data

Module 15: Big Data Security and Governance

·       Security challenges in a distributed environment

·       Authentication and authorization with Kerberos

·       Data encryption at rest and in transit

·       Data governance frameworks and compliance

·       Practical session: Implementing basic security measures for a Hadoop cluster

Module 16: Cluster Management with YARN

·       YARN architecture and its resource management capabilities

·       Understanding resource allocation and scheduling

·       Monitoring and troubleshooting YARN applications

·       Best practices for managing a multi-tenant cluster

·       Practical session: Submitting a Spark application to a YARN cluster and monitoring its progress

Module 17: Performance Tuning and Optimization

·       Identifying and resolving performance bottlenecks in Spark

·       Understanding Spark's Catalyst Optimizer and Tungsten engine

·       Techniques for caching, partitioning, and broadcasting data

·       Performance monitoring with Spark UI

·       Practical session: Optimizing a slow-running Spark job by applying tuning techniques

Module 18: End-to-End Big Data Analytics Project

·       Defining a project scope and requirements

·       Building a complete data pipeline from ingestion to visualization

·       Integrating Spark, Hadoop, and other tools

·       Presenting the final analysis and findings

·       Practical session: A comprehensive hands-on project that ties together all the modules learned, from data ingestion to advanced analytics

Requirements:

·       Participants should be reasonably proficient in English.

·       Applicants must live up to Armstrong Global Institute admission criteria.

Terms and Conditions

1. Discounts: Organizations sponsoring Four Participants will have the 5th attend Free

2. What is catered for by the Course Fees: Fees cater for all requirements for the training – Learning materials, Lunches, Teas, Snacks and Certification. All participants will additionally cater for their travel and accommodation expenses, visa application, insurance, and other personal expenses.

3. Certificate Awarded: Participants are awarded Certificates of Participation at the end of the training.

4. The program content shown here is for guidance purposes only. Our continuous course improvement process may lead to changes in topics and course structure.

5. Approval of Course: Our Programs are NITA Approved. Participating organizations can therefore claim reimbursement on fees paid in accordance with NITA Rules.

Booking for Training

Simply send an email to the Training Officer on training@armstrongglobalinstitute.com and we will send you a registration form. We advise you to book early to avoid missing a seat to this training.

Or call us on +254720272325 / +254725012095 / +254724452588

Payment Options

We provide 3 payment options, choose one for your convenience, and kindly make payments at least 5 days before the Training start date to reserve your seat:

1. Groups of 5 People and Above – Cheque Payments to: Armstrong Global Training & Development Center Limited should be paid in advance, 5 days to the training.

2. Invoice: We can send a bill directly to you or your company.

3. Deposit directly into Bank Account (Account details provided upon request)

Cancellation Policy

1. Payment for all courses includes a registration fee, which is non-refundable, and equals 15% of the total sum of the course fee.

2. Participants may cancel attendance 14 days or more prior to the training commencement date.

3. No refunds will be made 14 days or less before the training commencement date. However, participants who are unable to attend may opt to attend a similar training course at a later date or send a substitute participant provided the participation criteria have been met.

Tailor Made Courses

This training course can also be customized for your institution upon request for a minimum of 5 participants. You can have it conducted at our Training Centre or at a convenient location. For further inquiries, please contact us on Tel: +254720272325 / +254725012095 / +254724452588 or Email training@armstrongglobalinstitute.com

Accommodation and Airport Transfer

Accommodation and Airport Transfer is arranged upon request and at extra cost. For reservations contact the Training Officer on Email: training@armstrongglobalinstitute.com or on Tel: +254720272325 / +254725012095 / +254724452588