The data engineering space is evolving. Here are the resources I collected for practical data engineering resource.
Last Updated: 2023–04–02
- The Master Book On Distributed System: Designing Data-Intensive Application
- The SQL Fundamental Book: T-SQL Querying (Developer Reference)
- OLAP/Data Warehouse Must Read: The Data Warehouse Toolkit (The Definitive Guide to Dimensional Modeling, 3rd Edition)
- In-depth on Python: Fluent Python: Clear, Concise, and Effective Programming 2nd Edition
- Learning From the Creator: Spark: The Definitive Guide: Big Data Processing Made Simple
- Workflow Management Tool: Data Pipelines with Apache Airflow
- All You Need to Know About Streaming Foundation: Streaming Systems: The What, Where, When, and How of Large-Scale Data Processing
- Understand Data Science: R for Data Science: Import, Tidy, Transform, Visualize, and Model Data
- Data Communication: Storytelling with Data: A Data Visualization Guide for Business Professionals
- Gain Knowledge on Cloud Infra: Kubernetes in Action
- The Philosophy of Data Engineering: Fundamentals of Data Engineering: Plan and Build Robust Data Systems
- Data Mesh: Delivering Data-Driven Value at Scale
Discover the Essential Reading List for Data Engineers: 10 Classic Books You Can't Miss. While many free online resources are available, they often lack the ...
Data Engineering Space Leader (who to follow)
Practical Data Engineering Framework
Apache Airflow is undoubtedly the most popular open-source project for data engineering for years. It gains popularity at the right time with The Rise Of ...
Airflow has been widespread for years. Is Apache Airflow due for a replacement? mage-ai is the new ETL tool for data engineers to check out ...
Many modern data orchestration projects like Apache Airflow and Luigi are Python-based. Let's explore the popular data pipeline orchestration options for R.
- Druid: a high-performance, real-time analytics database that delivers sub-second queries on streaming and batch data at scale and under load.
- Trino: a query engine that runs at ludicrous speed
- ClickHouse: a column-oriented database that enables its users to generate powerful analytics, using SQL queries, in real-time.
- Classic Streaming Foundation (1): Streaming 101: The world beyond batch
- Classic Streaming Foundation (2): Streaming 102: The world beyond batch
- Spark Tuning Foundation (1): How-to: Tune Your Apache Spark Jobs (Part 1)
- Spark Tuning Foundation (2): How-to: Tune Your Apache Spark Jobs (Part 2)
- Airflow Scheduler Demystify: Airflow Scheduler 101
I hope my stories are helpful to you.
Personal budgeting APP like Mint/Personal Capital/Clarity only provide three limited types of charts. Have you ever wondered if charts are good enough to get better ...
Getting the work done isn't the journey's end. Your work should be your channel to get YOU self-promotion. I will give five tips to get ...
Using R and Shiny, we can build an app where the end users can interact with the data analysis we have done. I will show ...