Data Ingestion with Python Cookbook: A practical guide to ingesting, monitoring, and identifying errors in the data ingestion process 246252
Код товару: 246252Паперова книга
-
ISBN978-1837632602
-
Бренд
-
Автор
-
Рік2023
-
МоваАнглійська
-
ІлюстраціїЧорно-білі
Key Features
- Harness best practices to create a Python and PySpark data ingestion pipeline
- Seamlessly automate and orchestrate your data pipelines using Apache Airflow
- Build a monitoring framework by integrating the concept of data observability into your pipelines
Book Description
Data Ingestion with Python Cookbook offers a practical approach to designing and implementing data ingestion pipelines. It presents real-world examples with the most widely recognized open source tools on the market to answer commonly asked questions and overcome challenges.
You'll be introduced to designing and working with or without data schemas, as well as creating monitored pipelines with Airflow and data observability principles, all while following industry best practices. The book also addresses challenges associated with reading different data sources and data formats. As you progress through the book, you'll gain a broader understanding of error logging best practices, troubleshooting techniques, data orchestration, monitoring, and storing logs for further consultation.
By the end of the book, you'll have a fully automated set that enables you to start ingesting and monitoring your data pipeline effortlessly, facilitating seamless integration with subsequent stages of the ETL process.
What you will learn
- Implement data observability using monitoring tools
- Automate your data ingestion pipeline
- Read analytical and partitioned data, whether schema or non-schema based
- Debug and prevent data loss through efficient data monitoring and logging
- Establish data access policies using a data governance framework
- Construct a data orchestration framework to improve data quality
Who this book is for
This book is for data engineers and data enthusiasts seeking a comprehensive understanding of the data ingestion process using popular tools in the open source community. For more advanced learners, this book takes on the theoretical pillars of data governance while providing practical examples of real-world scenarios commonly encountered by data engineers
About the Author
Glaucia Esppenchutz is a data engineer with expertise in managing data pipelines and vast amounts of data using cloud and on-premises technologies. She worked in companies such as Globo, BMW Group, and Cloudera. Currently, she works at AiFi, specializing in the field of data operations for autonomous systems.
She comes from the biomedical field and shifted her career ten years ago to chase the dream of working closely with technology and data. She is in constant contact with the open source community, mentoring people and helping to manage projects, and has collaborated with the Apache, PyLadies group, FreeCodeCamp, Udacity, and MentorColor communities.
1'200 ₴
Купити
Monobank
от 135 ₴ / міс.
до 10 платежей
Покупка частинами Monobank
₴ / місяць
Для використання функції «Покупка частинами» необхідно мати картку Monobank.
Розділивши оплату на певну кількість платежів (від 3 до 10),
ви платите лише одну частину. Решта – раз на місяць списуватиметься з вашої карти.
Послуга може бути використана при замовлення на суму від 600 грн.
Увага! При покупці частинами знижки на товари не враховуються.
Щоб скористатися цією функцією, додайте в кошик товарів на суму від 600 грн.
На сторінці оформлення замовлення вкажіть спосіб оплати «Покупка частинами Monobank». Підтвердьте покупку у програмі Monobank.
-
Нова ПоштаБезкоштовно від
3'000,00 ₴ -
УкрпоштаБезкоштовно від
1'000,00 ₴ -
Meest ПоштаБезкоштовно від
3'000,00 ₴
Інші книги Packt Publishing
278861
1'400 ₴
263517
1'700 ₴
302705
1'600 ₴
263204
1'500 ₴
295064
1'600 ₴
305807
1'900 ₴
Характеристики
- Бренд
- Автор
- КатегоріяПрограмування
- Рік2023
- Сторінок414
- Формат165х235 мм
- ОбкладинкаМ'яка
- Тип паперуОфсетний
- МоваАнглійська
- ІлюстраціїЧорно-білі
Від видавця
Key Features
- Harness best practices to create a Python and PySpark data ingestion pipeline
- Seamlessly automate and orchestrate your data pipelines using Apache Airflow
- Build a monitoring framework by integrating the concept of data observability into your pipelines
Book Description
Data Ingestion with Python Cookbook offers a practical approach to designing and implementing data ingestion pipelines. It presents real-world examples with the most widely recognized open source tools on the market to answer commonly asked questions and overcome challenges.
You'll be introduced to designing and working with or without data schemas, as well as creating monitored pipelines with Airflow and data observability principles, all while following industry best practices. The book also addresses challenges associated with reading different data sources and data formats. As you progress through the book, you'll gain a broader understanding of error logging best practices, troubleshooting techniques, data orchestration, monitoring, and storing logs for further consultation.
By the end of the book, you'll have a fully automated set that enables you to start ingesting and monitoring your data pipeline effortlessly, facilitating seamless integration with subsequent stages of the ETL process.
What you will learn
- Implement data observability using monitoring tools
- Automate your data ingestion pipeline
- Read analytical and partitioned data, whether schema or non-schema based
- Debug and prevent data loss through efficient data monitoring and logging
- Establish data access policies using a data governance framework
- Construct a data orchestration framework to improve data quality
Who this book is for
This book is for data engineers and data enthusiasts seeking a comprehensive understanding of the data ingestion process using popular tools in the open source community. For more advanced learners, this book takes on the theoretical pillars of data governance while providing practical examples of real-world scenarios commonly encountered by data engineers
About the Author
Glaucia Esppenchutz is a data engineer with expertise in managing data pipelines and vast amounts of data using cloud and on-premises technologies. She worked in companies such as Globo, BMW Group, and Cloudera. Currently, she works at AiFi, specializing in the field of data operations for autonomous systems.
She comes from the biomedical field and shifted her career ten years ago to chase the dream of working closely with technology and data. She is in constant contact with the open source community, mentoring people and helping to manage projects, and has collaborated with the Apache, PyLadies group, FreeCodeCamp, Udacity, and MentorColor communities.
Зміст
Table of Contents
- Introduction to Data Ingestion
- Principals of Data Access – Accessing your Data
- Data Discovery – Understanding Our Data Before Ingesting It
- Reading CSV and JSON Files and Solving Problems
- Ingesting Data from Structured and Unstructured Databases
- Using PySpark with De?ned and Non-De?ned Schemas
- Ingesting Analytical Data
- Designing Monitored Data Workflows
- Putting Everything Together with Air?ow
- Logging and Monitoring Your Data Ingest in Airflow
- Automating Your Data Ingestion Pipelines
- Using Data Observability for Debugging, Error Handling, and Preventing Downtime
Data Ingestion with Python Cookbook: A practical guide to ingesting, monitoring, and identifying errors in the data ingestion process
1'200 ₴
Персонально для вас
259260
246261
1'000 ₴
245903
1'100 ₴
246276
1'100 ₴
264111
1'400 ₴
279443
1'400 ₴
Відгуки про Data Ingestion with Python Cookbook: A practical guide to ingesting, monitoring, and identifying errors in the data ingestion process