Overview
- Covers migrating from Pandas, Scikit-Learn to PySpark, from single-node to large-scale computing
- Explains deploying ML models to production with Scikit-Learn and PySpark
- Explains how to use PySpark for distributed machine learning
Access this book
Tax calculation will be finalised at checkout
Other ways to access
Table of contents (18 chapters)
Keywords
About this book
Migrate from pandas and scikit-learn to PySpark to handle vast amounts of data and achieve faster data processing time. This book will show you how to make this transition by adapting your skills and leveraging the similarities in syntax, functionality, and interoperability between these tools.
Distributed Machine Learning with PySpark offers a roadmap to data scientists considering transitioning from small data libraries (pandas/scikit-learn) to big data processing and machine learning with PySpark. You will learn to translate Python code from pandas/scikit-learn to PySpark to preprocess large volumes of data and build, train, test, and evaluate popular machine learning algorithms such as linear and logistic regression, decision trees, random forests, support vector machines, Naïve Bayes, and neural networks.
After completing this book, you will understand the foundational concepts of data preparation and machine learning and will have the skills necessary toapply these methods using PySpark, the industry standard for building scalable ML data pipelines.
What You Will Learn
- Master the fundamentals of supervised learning, unsupervised learning, NLP, and recommender systems
- Understand the differences between PySpark, scikit-learn, and pandas
- Perform linear regression, logistic regression, and decision tree regression with pandas, scikit-learn, and PySpark
- Distinguish between the pipelines of PySpark and scikit-learn
Who This Book Is For
Data scientists, data engineers, and machine learning practitioners who have some familiarity with Python, but who are new to distributed machine learning and the PySpark framework.Authors and Affiliations
About the author
Abdelaziz Testas, Ph.D., is a data scientist with over a decade of experience in data analysis and machine learning, specializing in the use of standard Python libraries and Spark distributed computing. He holds a Ph.D. in Economics from Leeds University and a Master's degree in Finance from Glasgow University. He has also earned several certificates in computer science and data science.
In the last ten years, he has worked for Nielsen in Fremont, California as a Lead Data Scientist focused on improving the company’s audience measurement through planning, initiating, and executing end-to-end data science projects and methodology work. He has created advanced solutions for Nielsen’s digital ad and content rating products by leveraging subject matter expertise in media measurement and data science. He is passionate about helping others improve their machine learning skills and workflows, and is excited to share his knowledge and experience with a wider audience through this book.
Bibliographic Information
Book Title: Distributed Machine Learning with PySpark
Book Subtitle: Migrating Effortlessly from Pandas and Scikit-Learn
Authors: Abdelaziz Testas
DOI: https://doi.org/10.1007/978-1-4842-9751-3
Publisher: Apress Berkeley, CA
eBook Packages: Professional and Applied Computing, Apress Access Books, Professional and Applied Computing (R0)
Copyright Information: Abdelaziz Testas 2023
Softcover ISBN: 978-1-4842-9750-6Published: 24 November 2023
eBook ISBN: 978-1-4842-9751-3Published: 23 November 2023
Edition Number: 1
Number of Pages: XX, 490
Number of Illustrations: 8 b/w illustrations
Topics: Artificial Intelligence, Machine Learning, Python