Generated by All in One SEO v5.0.1.1, this is an llms.txt file, used by LLMs to index the site. # Chengzhi Zhao Data Engineering Content Creator ## Sitemaps - [XML Sitemap](https://chengzhizhao.com/sitemap.xml): Contains all public & indexable URLs for this website. ## Posts - [How I Built a Tool to Visualize Expense In Sankey Diagram](https://chengzhizhao.com/how-i-built-a-tool-to-visualize-expense-in-sankey-diagram/) - My main goal is to enable people without programming experience to use the powerful Sankey Diagram by simply uploading the transaction CVS file from the popular site Mint.com. - [Stop Breaking Production Data Pipeline: Implementing Write-Audit-Publish (WAP) with Spark and Apache Iceberg](https://chengzhizhao.com/stop-breaking-production-data-pipeline-implementing-write-audit-publish-wap-with-spark-and-apache-iceberg/) - Learn how to implement the Write-Audit-Publish (WAP) pattern in your data pipelines using Apache Iceberg branching and PySpark to guarantee data quality. - [The Data Modeling Wars: Inmon vs. Kimball vs. Data Vault](https://chengzhizhao.com/the-data-modeling-wars-inmon-vs-kimball-vs-data-vault/) - Confused by data modeling? We break down the key differences between Inmon, Kimball, and Data Vault architectures so you can choose the right strategy for your data warehouse. - [Apache Spark 4.1 is Here: The Next Chapter in Unified Analytics](https://chengzhizhao.com/apache-spark-4-1-is-here-the-next-chapter-in-unified-analytics/) - Apache Spark 4.1 is here. Discover how Real-Time Mode (RTM), Declarative Pipelines, and Arrow-Native UDFs are transforming data engineering and PySpark performance - [Data Processing Guarantees Explained: Exactly-Once, At-Least-Once, and At-Most-Once](https://chengzhizhao.com/data-processing-guarantees-explained-exactly-once-at-least-once-and-at-most-once/) - Learn the difference between data processing guarantees (At-Most-Once, At-Least-Once, Exactly-Once) with simple real-world examples. Perfect for data engineering beginners - [2025 Retrospective: How AI Changed the Way I Engineer](https://chengzhizhao.com/2025-retrospective-how-ai-changed-the-way-i-engineer/) - 2025 marked the shift from experimenting with AI to relying on it. In this retrospective, I explore how AI killed the 'tedious task' but failed the 'context test'—specifically sharing why OpenAI, Claude, and Gemini all couldn't fix a complex protoc dependency that still required a human engineer. - [The Ultimate Apache Spark Guide: Performance Tuning, PySpark Examples, and New 4.0 Features](https://chengzhizhao.com/the-ultimate-apache-spark-guide-performance-tuning-pyspark-examples-and-new-4-0-features/) - The ultimate guide to Apache Spark. Learn performance tuning with PySpark examples, fix common issues like data skew, and explore new Spark 4.0 features. - [10 Best Books on Data Analytics with AI Agents – Read Before You Build!](https://chengzhizhao.com/10-best-books-on-data-analytics-with-ai-agents-read-before-you-build/) - Looking for the best books on data analytics and AI agents? Discover top-rated titles with summaries, user reviews, and expert recommendations for every data enthusiast and AI innovator. - [What Are Apache Flink Watermarks? A Beginner’s Guide to Handling Late Arrival Data](https://chengzhizhao.com/what-are-apache-flink-watermarks-a-beginners-guide-to-handling-late-arrival-data/) - Struggling with late or out-of-order data? Learn how Apache Flink Watermarks work with event time to build accurate, reliable real-time stream processing systems. - [Data Engineering Heats Up in June 2025: A Look at the Latest Developments](https://chengzhizhao.com/data-engineering-heats-up-in-june-2025-a-look-at-the-latest-developments/) - Stay current with the essential data engineering news from June 2025. This monthly roundup covers the biggest announcements from Databricks' Data + AI Summit, new Snowflake features, Apache Flink updates, and the growing role of AI and Apache Iceberg in the data landscape. - [Automate Social Media Like a Pro (Almost Free): Using n8n + DeepSeek AI](https://chengzhizhao.com/automate-social-media-like-a-pro-almost-free-using-n8n-deepseek-ai/) - Learn how to build a powerful, low-cost AI social media scheduler using n8n and DeepSeek. Automate content creation, shorten links, and schedule Twitter posts—without paying for Buffer, Hootsuite, or ChatGPT - [Deep Dive into Handling Apache Spark Data Skew](https://chengzhizhao.com/deep-dive-into-handling-apache-spark-data-skew/) - "Why my Spark job is running slow?" is an inevitable question. We will cover how to identify Spark data skew and how to handle data skew with different options, including key salting - [LLM for Data Visualization: How AI Shapes the Future of Analytics](https://chengzhizhao.com/llm-for-data-visualization-how-ai-shapes-the-future-of-analytics/) - Discover how to utilise LLM for data visualization by generating SQL queries using LLMs and building charts with Seaborn and Plotly. Learn how AI agents transform EDA and analysis. - [Boosting Spark Union Operator Performance: Optimization Tips for Improved Query Speed](https://chengzhizhao.com/boosting-spark-union-operator-performance-optimization-tips-for-improved-query-speed/) - We will focus on the Apache Spark Union Operator Performance with examples, show you the physical query plan, and share techniques for optimization in this story. - [4 Faster Pandas Alternatives for Data Analysis](https://chengzhizhao.com/4-faster-pandas-alternatives-for-data-analysis/) - Pandas is no doubt one of the most popular libraries in Python. However, Pandas doesn't shine in the land of data processing with a large dataset. We will compare 4 faster pandas alternatives for data analysis: Polars, Dask, Vaex, Modin - [DuckDB Local UI is Awesome!](https://chengzhizhao.com/duckdb-local-ui-is-awesome/) - Discover how DuckDB Local UI revolutionises your data exploration experience. After years of using external tools, DuckDB’s native interface provides a seamless, quick, and intuitive way to interact with your data projects - [The AI Wake-Up Call for Data Engineers: Why LLMs + MCP Matter Now](https://chengzhizhao.com/the-ai-wake-up-call-for-data-engineers-why-llms-mcp-matter-now/) - AI isn't coming for data engineering — it's becoming part of it. In this post, I explore how Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and Model Context Protocol (MCP) are transforming how data engineers build, query, and integrate modern data systems. Real-world tools like Cline, Cursor, and DuckDB show that the AI future of data engineering is already here. - [Data Engineering in 2025: A Practical Guide for New Grads Entering the AI-First Era](https://chengzhizhao.com/data-engineering-in-2025-a-practical-guide-for-new-grads-entering-the-ai-first-era/) - Explore how AI in data engineering is shaping the future. This 2025 guide helps new grads build the skills, tools, and mindset to thrive in a cloud-driven, AI-first world. - [Don’t Get Tripped Up! 10 Common Data Engineering Pitfalls](https://chengzhizhao.com/dont-get-tripped-up-10-common-data-engineering-pitfalls/) - Learn how to avoid 10 common data engineering pitfalls—like Spark data skew, Airflow retry chaos, schema drift, and more—with practical solutions - [How to Build an AI Agent for Data Analytics Without Writing SQL](https://chengzhizhao.com/how-to-build-an-ai-agent-for-data-analytics-without-writing-sql/) - We will demonstrate the construction of an AI agent from inception using LangChain and DuckDB. We can address business inquiries without SQL expertise. - [Beyond Basic Prompts: LLM + MCP Tackling Real-World Challenges—The Airflow 3.0 Auto-Update Example](https://chengzhizhao.com/beyond-basic-prompts-llm-mcp-tackling-real-world-challenges-the-airflow-3-0-auto-update-example/) - Learn how LLM + MCP synergy revolutionizes complex tasks. An Apache Airflow 3.0 case study demonstrates auto-updating DAGs and overcoming AI limitations. - [Unboxing Apache Airflow 3.0: What’s New, What’s Gone, and Why It Matters](https://chengzhizhao.com/unboxing-apache-airflow-3-0-whats-new-whats-gone-and-why-it-matters/) - Explore the latest features, UI updates, and key changes in Apache Airflow 3.0. This deep dive covers DAG versioning, event-driven scheduling, Docker setup, and more for data engineers and workflow automation pros. - [DeepSeek SmallPond: A Game-Changer for Data Engineers Seeking Lightweight Solutions](https://chengzhizhao.com/deepseek-smallpond-a-game-changer-for-data-engineers-seeking-lightweight-solutions/) - DeepSeek SmallPond is here to shake up data engineering. See how this lightweight open-source framework offers a fresh alternative to Apache Spark and Flink for batch and streaming processes. - [The Essential Reading List for Data Engineers: 10 Classic Books You Can't Miss](https://chengzhizhao.com/the-essential-reading-list-for-data-engineers-10-classic-books-you-cant-miss/) - Discover the Essential Reading List for Data Engineers: 10 Classic Books You Can't Miss. While many free online resources are available, they often lack the depth and context needed to truly master the field. In this article, I will share ten classic books that cover everything from fundamental technical skills like Python and SQL, to more advanced topics like Apache Spark, Apache Flink, Apache Beam, Apache Airflow, Kubernetes, distributed systems, and dimensional modeling. - [Apache Airflow 3.0 Is Coming Soon: Here’s What You Can Expect](https://chengzhizhao.com/apache-airflow-3-0-is-coming-soon-heres-what-you-can-expect/) - Discover the upcoming features in Apache Airflow 3.0, with insights from the Airflow 3.0 workstream. Get ready for the next big release! - [Uncovering the Truth About Apache Spark Performance: coalesce(1) vs. repartition(1)](https://chengzhizhao.com/uncovering-the-truth-about-apache-spark-performance-coalesce1-vs-repartition1/) - We will discuss a neglected part of Apache Spark Performance between coalesce(1) and repartition(1), and it could be one of the things to be attentive to when you check the Spark job performance. - [How to build a web crawler with MWAA (AWS Airflow) with CDK](https://chengzhizhao.com/how-to-build-a-web-crawler-with-mwaa-aws-airflow-with-cdk/) - Integrating Apache Airflow with the AWS ecosystem has become easier than ever with MWAA. To make MWAA work efficiently, I prepared a comprehensive guide using CDK to spin up MWAA and some tips for MWAA specifically to help you understand the deployment for Airflow in AWS. - [The Foundation of Data Validation](https://chengzhizhao.com/the-foundation-of-data-validation/) - If you are reading this blog post, you may have faced the challenge of data validation before, or you might be struggling with it. My goal in this post is to share my experience with data validation - [5 Lessons I Learned From a Totaled Car Accident](https://chengzhizhao.com/5-lessons-i-learned-from-a-totaled-car-accident/) - Experiencing a totaled car accident that results in the total loss of your vehicle is a difficult situation to deal with. I want to share what I learned with more people. - [Airflow Schedule Interval 101](https://chengzhizhao.com/airflow-schedule-interval-101/) - The airflow schedule interval could be a challenging concept to comprehend, even for developers work on Airflow for a while find difficult to grasp. A confusing question arises every once a while on StackOverflow is "Why my DAG is not running as expected?". This problem usually indicates a misunderstanding among the Airflow schedule interval. - [Bidding War on Housing Market? Let’s Use R For Exploratory Data Analysis](https://chengzhizhao.com/bidding-war-on-housing-market-lets-use-r-for-exploratory-data-analysis/) - Exploratory Data Analysis (EDA) is a methodology in data science as the initial approach to gain insights by visualizing and summarizing data. We will use some exploratory data analysis technics to find the reason behind the bidding war on the housing market. - [Visualizing Data with ggridges: Techniques to Eliminate Density Plot Overlaps in ggplot2](https://chengzhizhao.com/visualizing-data-with-ggridges-techniques-to-eliminate-density-plot-overlaps-in-ggplot2/) - When it comes to visualizing data with a histogram and dealing with multiple groups, it can be quite challenging. I have recently come across a useful ggplot2 extension called ggridges that has been helpful for my data exploratory tasks. - [Unlocking the Secrets of Slowly Changing Dimension (SCD): A Comprehensive View of 8 Types](https://chengzhizhao.com/unlocking-the-secrets-of-slowly-changing-dimension-scd-a-comprehensive-view-of-8-types/) - Slowly Changing Dimension (SCD) is critical to dimensional modeling. We will discuss the eight types of SCDs. By the end, you will clearly understand each type and be able to differentiate between SCDs in dimensional modeling. - [Demystifying Null in SQL: A Comprehensive Guide for Data Professionals](https://chengzhizhao.com/demystifying-null-in-sql-a-comprehensive-guide-for-data-professionals/) - Sometimes writing SQL can be frustrating, especially when encountering NULL values. This article can help you better understand NULL in SQL - [Data Engineering: Why It's About Much More Than Just the Tools You Use](https://chengzhizhao.com/data-engineering-why-its-about-much-more-than-just-the-tools-you-use/) - One key learning I had while chasing the latest tool is: Tools are great, but many data engineering problems cannot be resolved by using the newest tool but by human — Data Engineers. I want to share my thoughts on Why data engineering is about much more than just the tools you use. - [Building Better Data Warehouses with Dimensional Modeling: A Guide for Data Engineers](https://chengzhizhao.com/building-better-data-warehouses-with-dimensional-modeling-a-guide-for-data-engineers/) - Let's bring the data community's attention to the essential- Building Better Data Warehouses with Dimensional Modeling: A Guide for Data Engineers. - [Why R for Data Engineering is More Powerful Than You Thought](https://chengzhizhao.com/why-r-for-data-engineering-is-more-powerful-than-you-thought/) - R could add potential benefits to help the data engineering community. Let's discuss about Why R for Data Engineering is More Powerful Than You Thought. - [5 Hidden Apache Spark Facts That Fewer People Talk About](https://chengzhizhao.com/5-hidden-apache-spark-facts-that-fewer-people-talk-about/) - I want to share 5 hidden facts about Apache Spark that I learned throughout my career. Those can be helpful to you to save you some time reading the Apache Spark source code. - [The Practical Data Engineering Resource](https://chengzhizhao.com/the-practical-data-engineering-resource/) - The data engineering space is evolving. Here are the resources I collected for practical data engineering resource. - [How to Find the Best Deals On Time with R and Mage](https://chengzhizhao.com/how-to-find-the-best-deals-on-time-with-r-and-mage/) - How to find the best deals and coupons promptly can save you money and time. We can quickly build a weekend project that automatically finds the best deals on time with R and Mage - [4 Free Fantastic Diagramming Tools To Make Yours Stand Out](https://chengzhizhao.com/4-free-fantastic-diagramming-tools-to-make-yours-stand-out/) - Many diagrams bring less excitement to work with and view as the final result. I will share 4 free fantastic diagramming tools to make yours stand out. - [Get Fluent in Python Decorators by Visualizing It](https://chengzhizhao.com/get-fluent-in-python-decorators-by-visualizing-it/) - Python decorator is syntactic sugar. You can achieve everything without explicitly using the decorator. However, Using the decorator can help your code be more concise and readable. Ultimately, you write fewer lines of code by leveraging Python decorators. - [How to Engage with Users By Storytelling: Show Data Analytics in R and Shiny](https://chengzhizhao.com/how-to-engage-with-users-by-storytelling-show-data-analytics-in-r-and-shiny/) - Using R and Shiny, we can build an app where the end users can interact with the data analysis we have done. I will show you how to engage with users by storytelling - show data analytics in R and Shiny. - [Here Is What I Learned Using Apache Airflow over 6 Years](https://chengzhizhao.com/here-is-what-i-learned-using-apache-airflow-over-6-years/) - Apache Airflow is undoubtedly the most popular open-source project for data engineering for years. It gains popularity at the right time with The Rise Of Data Engineer. Today, I want to share my journey with Airflow and what I learned over 6 years. - [R For Data Analysis: How to Find the Perfect Cocomelon Video for Your Kids](https://chengzhizhao.com/r-for-data-analysis-how-to-find-the-perfect-cocomelon-video-for-your-kids/) - I will share my journey on using R for Data Analysis: building an end-to-end solution for exploring trending Cocomelon videos using R from scratch. - [Is ChatGPT Making People Lose Interest in Writing: Learning From Using ChatGPT](https://chengzhizhao.com/is-chatgpt-making-people-lose-interest-in-writing-learning-from-using-chatgpt/) - ChatGPT is powerful and scary. As people interested in writing, we have thoughts and manually type each word. Will this change how we write, and will more people lose interest? After using ChatGPT for some time, my answer is: No. It isn't capable of changing anything humans produce. But it could hurt people who want to get started. - [Streaming Data Is Exciting: What You Need to Know Before Taking the Plunge](https://chengzhizhao.com/streaming-data-is-exciting-what-you-need-to-know-before-taking-the-plunge/) - Is streaming data necessary for this particular use case? Rather than blindly diving in, it's essential first to acknowledge the realities of working with streaming data. - [Mastering Gantt Charts: Learn How to Build Them Using Code Alone](https://chengzhizhao.com/mastering-gantt-charts-learn-how-to-build-them-using-code-alone/) - Learn how to master Gantt charts using code alone in project management tracking. Many UI-based tools for Gantt chart creation can take a lot of work to manage. Discover the elegant and practical approach of building Gantt charts with code to reflect fast-paced project changes in this insightful story. - [Think In SQL – Window Function Demisify](https://chengzhizhao.com/think-in-sql-window-function-demisify/) - Understanding window function is critical for anyone that writes SQL daily. In this story, let's think in SQL and demystify the window function with examples and diagrams. - [6 Side Project Ideas for New and Experienced Data Engineers](https://chengzhizhao.com/6-side-project-ideas-for-new-and-experienced-data-engineers/) - Data engineers can work on some side projects to get experience. Those projects could initiate impressive discussions to help you land a dream job. We will introduce 6 data engineering side project ideas regardless of your experience. - [I Built a Game By Using Streaming Data: A Fun Way for Data Visualization](https://chengzhizhao.com/i-built-a-game-by-using-streaming-data-a-fun-way-for-data-visualization/) - Data visualization has always been a delightful area for me to work as a data professional. Visualizing data is like an art. Can I visualize streaming data in another way? I built a game by using streaming data, and this is a fun way for data visualization. - [Think in SQL — Avoid Writing SQL in a Top to Bottom Approach](https://chengzhizhao.com/think-in-sql-avoid-writing-sql-in-a-top-to-bottom-approach/) - SQL logical query processing order can help you understand why to change writing SQL in the top to bottom approach. It can also help you think in SQL clearly and develop your query more effectively - [How I Found Peace of Mind After Timeboxing](https://chengzhizhao.com/how-i-found-peace-of-mind-after-timeboxing/) - I am tired of continuous rapid context switching and keep being distracted. Interruption kept occurring and continued the next day. Timeboxing helped me find peace of mind in an isolated environment to concentrate on my task. - [5 Fantastic Data Pipeline Orchestration Tools For R](https://chengzhizhao.com/5-fantastic-data-pipeline-orchestration-tools-for-r/) - Many modern data orchestration projects like Apache Airflow and Luigi are Python-based. Let's explore the popular data pipeline orchestration options for R. - [How to Build Data Animation in R](https://chengzhizhao.com/how-to-build-data-animation-in-r/) - Have you seen any beautiful racing bar chart data animation on Youtube and wondered how it was built? I will show you how to use gganimate in R to animate data by creating a racing bar chart as an example. - [Is Apache Airflow Due for Replacement? The First Impression Of mage-ai](https://chengzhizhao.com/is-apache-airflow-due-for-replacement-the-first-impression-of-mage-ai/) - Airflow has been widespread for years. Is Apache Airflow due for a replacement? mage-ai is the new ETL tool for data engineers to check out as a substitution. I have taken a first impression of mage-ai and will share my thoughts. - [How to Visualize Monthly Expenses in a Comprehensive Way: Develop a Sankey Diagram in R](https://chengzhizhao.com/how-to-visualize-monthly-expenses-in-a-comprehensive-way-develop-a-sankey-diagram-in-r/) - Personal budgeting APP like Mint/Personal Capital/Clarity only provide three limited types of charts. Have you ever wondered if charts are good enough to get better ideas on your monthly income and expense? Are there ways to visualize monthly expenses in a comprehensive way? In this article, I will share with you how to create a Sankey Diagram In R to better help you gain more insights into your personal financial situation. - [Interested In Becoming A Data Engineer? A Glimpse Of The Data Engineering Role](https://chengzhizhao.com/interested-in-becoming-a-data-engineer-a-glimpse-of-the-data-engineering-role/) - Interested In Becoming A Data Engineer? This article will show a glimpse of the data engineering role and the requirements to become a data engineer to help you make that decision. - [5 Tips for Self-Promotion as Data Professionals](https://chengzhizhao.com/5-tips-for-self-promotion-as-data-professionals/) - Getting the work done isn't the journey's end. Your work should be your channel to get YOU self-promotion. I will give five tips to get self-promotion as data professionals - [My Experience with Amazon Affiliate Program on Medium.com in 2023](https://chengzhizhao.com/my-experience-with-amazon-affiliate-program-on-medium-com-in-2023/) - Amazon Affiliate Program is the best pick as an affiliate marketing program. In this article, we will go through my experience with Amazon Affiliate Program and discuss if we can use it on Medium.com. - [Five Tips for Saving Money At Target](https://chengzhizhao.com/five-tips-for-saving-money-at-target/) - How to maximize your savings when you shop at Target? I will unveil 5 ways of saving money when you shop at Target to maximize your savings. - [My Life With Depersonalization Derealization Disorder (DPDR)](https://chengzhizhao.com/my-life-with-depersonalization-derealization-disorder-dpdr/) - One of the challenges for me is identifying this blurry feeling. It took me years to find out the name of my issue. Five years ago, I randomly searched for my symptoms. I found Depersonalization Derealization Disorder ## Pages - [Home](https://chengzhizhao.com/) - Hi! I am Chengzhi Zhao. I write topics on data engineering & data science, productivity, and DIY projects. I am a contributor to Apache Airflow and Flink. I write blogs at ChengzhiZhao@Medium, and my personal blog is https://chengzhizhao.com/ - [Home Dark](https://chengzhizhao.com/home-page/) - Trusted Tech Stack Don't Just Build Pipelines.Architect Them. Pipeline Optimization Debug slow jobs, fix failures, and reduce costs. Deep-dive analysis of your existing Airflow/Spark setup. Real-Time Architecture Move from batch to low-latency streaming. Design and implement robust Flink and Kafka solutions. AI-Ready Infrastructure Prepare your platform for LLMs. Build RAG pipelines and vector database integrations. - [Privacy Policy](https://chengzhizhao.com/privacy-policy/) - Who we are Suggested text: Our website address is: https://chengzhizhao.com. Comments Suggested text: When visitors leave comments on the site we collect the data shown in the comments form, and also the visitor’s IP address and browser user agent string to help spam detection. An anonymized string created from your email address (also called a - [About](https://chengzhizhao.com/about/) - About Me Hi! I am Chengzhi Zhao. I am a data engineer who writes content on data engineering & data science, productivity, and DIY projects. I am a contributor to Apache Airflow and Apache Flink. I love sharing my experience on data-related topics. I write blogs at ChengzhiZhao@Medium, and my personal blog is https://chengzhizhao.com/.I hope ## Lasso - [Data Pipelines with Apache Airflow](https://chengzhizhao.com/amzn-data-pipelines-with-apache-airflow/) - [AI Engineering: Building Applications with Foundation Models](https://chengzhizhao.com/amzn-ai-engineering-building-applications-with-foundation-models/) - [Learning Spark: Lightning-Fast Data Analytics](https://chengzhizhao.com/amzn-learning-spark-lightning-fast-data-analytics/) - [Prompt Engineering for Generative AI: Future-Proof Inputs for Reliable AI Outputs](https://chengzhizhao.com/amzn-prompt-engineering-for-generative-ai-future-proof-inputs-for-reliable-ai-outputs/) - [The Data Warehouse Toolkit: The Definitive Guide to Dimensional Modeling](https://chengzhizhao.com/amzn-the-data-warehouse-toolkit-the-definitive-guide-to-dimensional-modeling/) - [Spark: The Definitive Guide: Big Data Processing Made Simple](https://chengzhizhao.com/amzn-spark-the-definitive-guide-big-data-processing-made-simple/) - [Building Agentic AI Systems: Create intelligent, autonomous AI agents that can reason, plan, and adapt](https://chengzhizhao.com/amzn-building-agentic-ai-systems-create-intelligent-autonomous-ai-agents-that-can-reason-plan-and-adapt/) - [Fluent Python: Clear, Concise, and Effective Programming](https://chengzhizhao.com/amzn-fluent-python-clear-concise-and-effective-programming/) - [DuckDB: Up and Running: Fast Data Analytics and Reporting](https://chengzhizhao.com/amzn-duckdb-up-and-running-fast-data-analytics-and-reporting/) - [Show Your Work!: 10 Ways to Share Your Creativity and Get Discovered (Austin Kleon)](https://chengzhizhao.com/amzn-show-your-work-10-ways-to-share-your-creativity-and-get-discovered-austin-kleon/) - [R for Data Science: Import, Tidy, Transform, Visualize, and Model Data](https://chengzhizhao.com/amzn-r-for-data-science-import-tidy-transform-visualize-and-model-data/) - [Mastering Shiny: Build Interactive Apps, Reports, and Dashboards Powered by R](https://chengzhizhao.com/amzn-mastering-shiny-build-interactive-apps-reports-and-dashboards-powered-by-r/) - [Kubernetes in Action](https://chengzhizhao.com/amzn-kubernetes-in-action/) - [Hands-On Large Language Models: Language Understanding and Generation](https://chengzhizhao.com/amzn-hands-on-large-language-models-language-understanding-and-generation/) - [Streaming Systems: The What, Where, When, and How of Large-Scale Data Processing](https://chengzhizhao.com/amzn-streaming-systems-the-what-where-when-and-how-of-large-scale-data-processing/) - [Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems](https://chengzhizhao.com/amzn-designing-data-intensive-applications-the-big-ideas-behind-reliable-scalable-and-maintainable-systems/) - [Practical Deep Learning for Cloud, Mobile, and Edge: Real-World AI & Computer-Vision Projects Using Python, Keras & TensorFlow](https://chengzhizhao.com/amzn-practical-deep-learning-for-cloud-mobile-and-edge-real-world-ai-computer-vision-projects-using-python-keras-tensorflow/) - [Stream Processing with Apache Flink: Fundamentals, Implementation, and Operation of Streaming Applications](https://chengzhizhao.com/amzn-stream-processing-with-apache-flink-fundamentals-implementation-and-operation-of-streaming-applications/) - [Deep Reinforcement Learning Hands-On: A practical and easy-to-follow guide to RL from Q-learning and DQNs to PPO and RLHF](https://chengzhizhao.com/amzn-deep-reinforcement-learning-hands-on-a-practical-and-easy-to-follow-guide-to-rl-from-q-learning-and-dqns-to-ppo-and-rlhf/) - [Amazon](https://chengzhizhao.com/amzn-amazon-18/) - [Amazon](https://chengzhizhao.com/amzn-amazon-17/) - [Sorry! Something went wrong!](https://chengzhizhao.com/amzn-sorry-something-went-wrong-3/) - [Sorry! Something went wrong!](https://chengzhizhao.com/amzn-sorry-something-went-wrong-2/) - [Page Not Found](https://chengzhizhao.com/amzn-page-not-found-4/) - [Page Not Found](https://chengzhizhao.com/amzn-page-not-found-3/) - [Page Not Found](https://chengzhizhao.com/amzn-page-not-found-2/) - [Page Not Found](https://chengzhizhao.com/amzn-page-not-found/) - [Sorry! Something went wrong!](https://chengzhizhao.com/amzn-sorry-something-went-wrong/) - [Data Science for Business: What You Need to Know about Data Mining and Data-Analytic Thinking](https://chengzhizhao.com/amzn-data-science-for-business-what-you-need-to-know-about-data-mining-and-data-analytic-thinking/) - [The AI-Powered Enterprise: Harness the Power of Ontologies to Make Your Business Smarter, Faster, and More Profitable](https://chengzhizhao.com/amzn-the-ai-powered-enterprise-harness-the-power-of-ontologies-to-make-your-business-smarter-faster-and-more-profitable/) - [Artificial Intelligence: A Guide for Thinking Humans](https://chengzhizhao.com/amzn-artificial-intelligence-a-guide-for-thinking-humans/) - [You Look Like a Thing and I Love You: How Artificial Intelligence Works and Why It's Making the World a Weirder Place](https://chengzhizhao.com/amzn-you-look-like-a-thing-and-i-love-you-how-artificial-intelligence-works-and-why-its-making-the-world-a-weirder-place/) - [Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems](https://chengzhizhao.com/amzn-hands-on-machine-learning-with-scikit-learn-keras-and-tensorflow-concepts-tools-and-techniques-to-build-intelligent-systems/) - [DuckDB in Action](https://chengzhizhao.com/amzn-amazon-3/) - [DuckDB in Action](https://chengzhizhao.com/amzn-amazon-4/) - [Polars Cookbook: Over 60 practical recipes to transform, manipulate, and analyze your data using Python Polars 1.x](https://chengzhizhao.com/amzn-polars-cookbook-over-60-practical-recipes-to-transform-manipulate-and-analyze-your-data-using-python-polars-1-x/) - [Python for Data Analysis: Data Wrangling with pandas, NumPy, and Jupyter](https://chengzhizhao.com/amzn-python-for-data-analysis-data-wrangling-with-pandas-numpy-and-jupyter/) - [Storytelling with Data: A Data Visualization Guide for Business Professionals](https://chengzhizhao.com/amzn-storytelling-with-data-a-data-visualization-guide-for-business-professionals-2/) - [DataStory: Explain Data and Inspire Action Through Story](https://chengzhizhao.com/amzn-datastory-explain-data-and-inspire-action-through-story/) - [Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones](https://chengzhizhao.com/amzn-atomic-habits-an-easy-proven-way-to-build-good-habits-break-bad-ones-2/) - [Amazon](https://chengzhizhao.com/amzn-amazon-16/) - [System Design Interview – An insider's guide](https://chengzhizhao.com/amzn-system-design-interview-an-insiders-guide/) - [System Design Interview – An Insider's Guide: Volume 2](https://chengzhizhao.com/amzn-system-design-interview-an-insiders-guide-volume-2/) - [Training Kit (Exam 70-461) Querying Microsoft SQL Server 2012 (MCSA) (Microsoft Press Training Kit)](https://chengzhizhao.com/amzn-training-kit-exam-70-461-querying-microsoft-sql-server-2012-mcsa-microsoft-press-training-kit/) - [Amazon](https://chengzhizhao.com/amzn-amazon-14/) - [Amazon](https://chengzhizhao.com/amzn-amazon-15/) - [Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones](https://chengzhizhao.com/amzn-atomic-habits-an-easy-proven-way-to-build-good-habits-break-bad-ones/) - [Amazon](https://chengzhizhao.com/amzn-amazon-12/) - [Amazon](https://chengzhizhao.com/amzn-amazon-13/) - [Amazon](https://chengzhizhao.com/amzn-amazon-10/) - [Amazon](https://chengzhizhao.com/amzn-amazon-11/) - [Amazon](https://chengzhizhao.com/amzn-amazon-8/) - [Amazon](https://chengzhizhao.com/amzn-amazon-9/) - [Amazon](https://chengzhizhao.com/amzn-amazon-7/) - [Amazon](https://chengzhizhao.com/amzn-amazon-6/) - [Amazon](https://chengzhizhao.com/amzn-amazon-5/) - [DuckDB: Up and Running: Fast Data Analytics and Reporting](https://chengzhizhao.com/amzn-duckdb-up-and-running-fast-data-analytics-and-reporting-2/) - [Build a Large Language Model (From Scratch)](https://chengzhizhao.com/amzn-build-a-large-language-model-from-scratch/) - [Amazon](https://chengzhizhao.com/amzn-amazon-2/) - [Amazon](https://chengzhizhao.com/amzn-amazon/) - [Data Mesh: Delivering Data-Driven Value at Scale](https://chengzhizhao.com/amzn-data-mesh-delivering-data-driven-value-at-scale/) - [Fundamentals of Data Engineering: Plan and Build Robust Data Systems](https://chengzhizhao.com/amzn-fundamentals-of-data-engineering-plan-and-build-robust-data-systems/) - [T-SQL Querying (Developer Reference)](https://chengzhizhao.com/amzn-t-sql-querying-developer-reference/) - [Storytelling with Data: A Data Visualization Guide for Business Professionals](https://chengzhizhao.com/amzn-storytelling-with-data-a-data-visualization-guide-for-business-professionals/) ## Categories - [Mental Health](https://chengzhizhao.com/category/blog/mental-health/) - [Writing](https://chengzhizhao.com/category/blog/writing/) - [Personal Finance](https://chengzhizhao.com/category/blog/personal-finance/) - [Data Engineering](https://chengzhizhao.com/category/blog/data-engineering/) - [Blog](https://chengzhizhao.com/category/blog/) - [Productivity](https://chengzhizhao.com/category/blog/productivity/) - [AI](https://chengzhizhao.com/category/blog/ai/) ## Tags - [Mental Health](https://chengzhizhao.com/tag/mental-health/) - [Depersonalization Derealization Disorder](https://chengzhizhao.com/tag/depersonalization-derealization-disorder/) - [DPDR](https://chengzhizhao.com/tag/dpdr/) - [Entrepreneurship](https://chengzhizhao.com/tag/entrepreneurship/) - [Writing](https://chengzhizhao.com/tag/writing/) - [Shopping](https://chengzhizhao.com/tag/shopping/) - [Deals](https://chengzhizhao.com/tag/deals/) - [Bargains](https://chengzhizhao.com/tag/bargains/) - [Data Engineering](https://chengzhizhao.com/tag/data-engineering/) - [Data](https://chengzhizhao.com/tag/data/) - [self-promotion](https://chengzhizhao.com/tag/self-promotion/) - [Apache Spark](https://chengzhizhao.com/tag/apache-spark/) - [Spark Performance](https://chengzhizhao.com/tag/spark-performance/) - [R](https://chengzhizhao.com/tag/r/) - [ggplot2](https://chengzhizhao.com/tag/ggplot2/) - [Sankey Diagram](https://chengzhizhao.com/tag/sankey-diagram/) - [Apache Airflow](https://chengzhizhao.com/tag/apache-airflow/) - [mage-ai](https://chengzhizhao.com/tag/mage-ai/) - [Side Projects](https://chengzhizhao.com/tag/side-projects/) - [Productivity](https://chengzhizhao.com/tag/productivity/) - [Deep Work](https://chengzhizhao.com/tag/deep-work/) - [Timeboxing](https://chengzhizhao.com/tag/timeboxing/) - [Data Visualization](https://chengzhizhao.com/tag/data-visualization/) - [Python](https://chengzhizhao.com/tag/python/) - [Data Pipeline](https://chengzhizhao.com/tag/data-pipeline/) - [Orchestration](https://chengzhizhao.com/tag/orchestration/) - [SQL](https://chengzhizhao.com/tag/sql/) - [Vaex](https://chengzhizhao.com/tag/vaex/) - [DuckDb](https://chengzhizhao.com/tag/duckdb/) - [Polars](https://chengzhizhao.com/tag/polars/) - [Modin](https://chengzhizhao.com/tag/modin/) - [Mermaid](https://chengzhizhao.com/tag/mermaid/) - [Gantt Charts](https://chengzhizhao.com/tag/gantt-charts/) - [Project Management](https://chengzhizhao.com/tag/project-management/) - [Apache Flink](https://chengzhizhao.com/tag/apache-flink/) - [Apache Beam](https://chengzhizhao.com/tag/apache-beam/) - [Data Streaming](https://chengzhizhao.com/tag/data-streaming/) - [Personal Development](https://chengzhizhao.com/tag/personal-development/) - [Data Analysis](https://chengzhizhao.com/tag/data-analysis/) - [Shiny](https://chengzhizhao.com/tag/shiny/) - [Diagramming](https://chengzhizhao.com/tag/diagramming/) - [Data Warehouse](https://chengzhizhao.com/tag/data-warehouse/) - [SCD](https://chengzhizhao.com/tag/scd/) - [Life](https://chengzhizhao.com/tag/life/) - [Scrapy](https://chengzhizhao.com/tag/scrapy/) - [AI](https://chengzhizhao.com/tag/ai/) - [LLM](https://chengzhizhao.com/tag/llm/) - [LangChain](https://chengzhizhao.com/tag/langchain/) - [LangGraph](https://chengzhizhao.com/tag/langgraph/) - [SmallPond](https://chengzhizhao.com/tag/smallpond/) - [Career](https://chengzhizhao.com/tag/career/) - [MCP](https://chengzhizhao.com/tag/mcp/) - [Flink](https://chengzhizhao.com/tag/flink/) - [Kafka](https://chengzhizhao.com/tag/kafka/) - [Data Modeling](https://chengzhizhao.com/tag/data-modeling/) - [Apache Iceberg](https://chengzhizhao.com/tag/apache-iceberg/)