arrow_back
Project Kick Start Session
Kafka Streaming Project Presentation
Kafka Streaming Project Curriculum by Clever Studies
Kafka Streaming Project Project Kick Start Session
Kafka Streaming Project Project Kick Start Session Q&A
Join Clever Studies Premium WhatsApp Group
[Sprint-1] Project Introduction
Project Curriculum Walkthrough (Part-1)
Project - Use Case Explanation (Part-2)
Project KPIs-Power BI Report Insights
Project - SDLC Phases (Part-3)
Agile Methodology (Part-4)
Project - Architecture Explanation (Part-5)
Azure DevOps Organization (ADO) Walkthrough (Part-6)
Sprint-1 Creation of Sprints and User Stories in ADO (Part-7)
Creation of Budget Alerts in Azure Account (Part-8)
Kafka Streaming Project Bonus
[Sprint-1] Resources
[Sprint-2] Creation of Azure resources (Infra Setup)
Sprint Review Call
Sprint-2 Agenda
Sprint-2 Creation of Sprints and User Stories in ADO (Part-1)
Creation of RSG and Vnet-Subnet, Databricks in Azure (Part-2)
Creation of Databricks Cluster (Part-3)
Creation of ADF and Setting up Azure Git Versioning (Part-4)
Creation of Storage Account-Key Vault-Logic App (Part-5)
Sprint-2 Project Infra Setup Bonus
[Sprint-2] Resources
[Sprint-3] Azure Resources Connection Establishment
Sprint-3 Agenda
Sprint-2 Review call
Integration Runtime-Linked Serices-Dataset (Part-1)
Managed Identity Vs Service Principal in Azure (Part-2)
System Assigned Managed Identity Vs User Assigned Managed Identity (Part-3)
Integratioin Runtime Setup Approach (Part-4)
Creating IR and Liked Services and SPNs (Part-5)
Creating Spark Cluster Configuration (Part-6)
Creating Secret Scope for Databricks Cluster Config (Part-7)
Sprint-3 Q&A-Bonus Session
Sprint-3 Resources
[Sprint-4] MySQL to Azure Data Ingestion
Sprint-3 Review
Sprint-4 Agenda
Linked Service and Dataset Parameters Understanding (Part-1)
Setting up source data in MySQL (Part-2)
Setting up a metadata table in Databricks (Part-3)
Linked Services Parameterization (Part-4)
Datasets Creation in Azure (Part-5)
ADF Pipeline development (Part-6)
Sprint-4 Q&A Bonus
Sprint-4 Resources
MYSQL Source Datasets
[Sprint-5] Getting Started with Kafka
Sprint-4 Review Call
Sprint-5 Agenda
In absence of Kafka (Part-1)
Kafka - Use case explained (Part-2)
Kafka Introduction (Part-3)
Kafka Cluster and Broker (Part-4)
Kafka Producer and Topic (Part-5)
Kafka Partition-Offset-Message (Part-6)
Apache Kafka Vs Kafka on Confluent (Part-7)
Sprint-5 Resources
[Sprint-6] Working with Confluent Kafka (Lab)
Sprint-5 Review Call
Working with Kafka Lab - Agenda
Confluent Workspace Walkthrough (Part-1)
Source data generation (Part-2)
Kafka Producer code to send data to Kafka Topic (Part-3)
Kafka Consumer code to subscribe data from Kafka Topic (Part-4)
CheckPointing Vs StartingOffest in Kafka Consumer Code (Part-5)
Sprint-6 Q&A
Sprint-6 Bonus
Sprint-6 Resources
[Sprint-7] Working with Autoloader and Delta Live Tables (Theory+Lab)
Sprint-7 Agenda
Sprint-6 Review Call
Overview of Databricks Autoloader and DLT (Part-1)
Autoloader Theory (Part-2)
Delta Live Tables (DLT) Theory (Part-3)
Autoloader and Delta Live Tables Q&A (Part-4)
Creation of Bronze Layer Tables with Autoloader (Part-5)
Creation of Silver Layer Tables in DLT (Part-6)
Creation of Materialized Views in Gold Layer (Part-7)
Creation of DLT Pipeline (Part-8)
Sprint-7 Bonus Q&A
Sprint-7 Resources
[Sprint-8] Implementation of Logging-Email Alerts-Triggers
Sprint-7 Review Call
Sprint-8 Agenda
Databricks Types of clusters (Part-1)
Logging Implementation in ADF Pipeline (Part-2)
Email Alerts Implementation in ADF Pipeline (Part-3)
Creating Triggers for ADF Pipeline (Part-4)
Sprint-8 Bonus Q&A
Sprint-8 Resources
[Sprint-9] Backlogs & Production Deployment
Sprint-8 Review Call
Sprint-9 Agenda
Overview of Kafka and DLT logging implementation (Part-1)
Kafka Producer Messages Acknowledgment - On Console (Part-2)
Kafka Producer Messages Acknowledgment - log file generation (Part-3)
Kafka Consumer and DLT logging Implementation (Part-4)
Kafka Consumer and DLT email Alerts (Part-5)
Prod Deployment Intro Theory (Part-6)
Prod Deployment - Databricks Deliverables (Part-7)
Prod Deployment - Bonus Q&A
Sprint-9 Resources
[Sprint-10] ADF-Production deployment & Project Q&A
Sprint-9 Review Call
Sprint-10 Agenda
Azure Git and ADO Pipeline Theory (Part-1)
ADO Pipeline Preparation (Part-2)
ADF Prod Deployment (Part-3)
Project Call Sheet Walkthrough (Part-4)
Realtime Project Execution Plan Explanation (Part-5)
Project Call Sheet Walkthrough (Part-6)
Sprint 10 - Bonus Q&A
Sprint-10 Resources