Practical Data Science on Databricks: Scaling end-to-end machine learning on Databricks

Take your machine learning skills to the next level by mastering databricks and building robust ML pipeline solutions for future ML innovations

Key Features

• Learn to build robust ML pipeline solutions for databricks transition
• Master commonly available features like AutoML and MLflow
• Leverage data governance and model deployment using MLflow model registry
• Purchase of the print or Kindle book includes a free PDF eBook

Book Description

Unleash the potential of databricks for end-to-end machine learning with this comprehensive guide, tailored for experienced data scientists and developers transitioning from DIY or other cloud platforms. Building on a strong foundation in Python, Practical Machine Learning on Databricks serves as your roadmap from development to production, covering all intermediary steps using the databricks platform.

You'll start with an overview of machine learning applications, databricks platform features, and MLflow. Next, you'll dive into data preparation, model selection, and training essentials and discover the power of databricks feature store for precomputing feature tables. You'll also learn to kickstart your projects using databricks AutoML and automate retraining and deployment through databricks workflows.

By the end of this book, you'll have mastered MLflow for experiment tracking, collaboration, and advanced use cases like model interpretability and governance. The book is enriched with hands-on example code at every step. While primarily focused on generally available features, the book equips you to easily adapt to future innovations in machine learning, databricks, and MLflow.

What you will learn

• Transition smoothly from DIY setups to databricks
• Master AutoML for quick ML experiment setup
• Automate model retraining and deployment
• Leverage databricks feature store for data prep
• Use MLflow for effective experiment tracking
• Gain practical insights for scalable ML solutions
• Find out how to handle model drifts in production environments

Who this book is for

This book is for experienced data scientists, engineers, and developers proficient in Python, statistics, and ML lifecycle looking to transition to databricks from DIY clouds. Introductory Spark knowledge is a must to make the most out of this book, however, end-to-end ML workflows will be covered. If you aim to accelerate your machine learning workflows and deploy scalable, robust solutions, this book is an indispensable resource.

Table of Contents

1. ML Process and Challenges
2. Overview of ML on Databricks
3. Utilizing Feature Store
4. Understanding MLflow Components
5. Create a Baseline Model for Bank Customer Churn Prediction Using AutoML
6. Model Versioning and Webhooks
7. Model Deployment Approaches
8. Automating ML Workflows Using the Databricks Jobs
9. Model Drift Detection for Our Churn Prediction Model and Retraining
10. CI/CD to Automate Model Retraining and Re-Deployment.