MLOps on Databricks for Senior Cloud Engineers
Master MLOps on Databricks: Automate ML pipelines, ensure model quality, and implement robust monitoring for production-ready ML systems.
...
Share
Databricks ML Environment Setup and MLflow Fundamentals
Unit 1: Databricks Workspace Configuration
MLOps Workspace Setup
Cluster Configuration
Databricks Connect
Unit 2: MLflow Fundamentals
MLflow Overview
MLflow Tracking Setup
Logging Parameters & Metrics
MLflow Models
MLflow Projects
MLflow Model Registry
Unit 3: Azure Integration and Security
Azure Key Vault Setup
Azure Container Registry
Access Control
Data Encryption
Network Security
Automated Model Training with Databricks Jobs and Delta Live Tables
Unit 1: Databricks Jobs for Model Training
Intro to Databricks Jobs
Creating Your First Job
Scheduling Jobs
Job Parameters
Error Handling
Unit 2: Delta Live Tables for Feature Engineering
Intro to Delta Live Tables
Creating a DLT Pipeline
Data Quality with DLT
Auto-Scaling DLT
DLT and Feature Stores
Unit 3: MLflow Tracking in Databricks Jobs
MLflow and Databricks
Logging Metrics
Artifact Logging
Experiment Management
MLflow Model Registry and Deployment Strategies
Unit 1: MLflow Model Registry Deep Dive
Model Registry Intro
Registering Your First Model
Model Versioning
Model Stages
Model Metadata
Unit 2: Deployment Strategies on Databricks
Batch Inference w/ Jobs
Real-time Inference Intro
Deploying to Model Serving
Model Serving Configuration
Custom Model Serving
Unit 3: Model Export and External Deployment
Model Export Overview
Export to Azure ML
Export to Kubernetes
Unit 4: Automating Deployment with CI/CD
CI/CD Pipeline Intro
Azure DevOps Integration
CI/CD for Machine Learning on Databricks
Unit 1: CI/CD Pipeline Setup for Databricks ML
Intro to ML CI/CD
Choosing Your CI/CD Tool
Connecting Databricks
Repo Structure
Unit 2: Automated Testing and Validation
Unit Testing ML Code
Integration Testing
Schema Validation
Data Quality Checks
Unit 3: Automated Model Deployment
Model Packaging
Batch Inference
Real-time Inference
Environment Promotion
Unit 4: CI/CD Pipeline Orchestration
Pipeline Definition
Triggering the Pipeline
Model Monitoring and Performance Management
Unit 1: Introduction to Model Monitoring on Databricks
Why Monitor Models?
Key Metrics to Track
Databricks Monitoring Tools
Unit 2: Implementing Model Monitoring with MLflow
Logging Metrics with MLflow
MLflow UI for Monitoring
Custom Dashboards
Unit 3: Detecting and Mitigating Model Drift
Understanding Model Drift
Detecting Data Drift
Detecting Concept Drift
Retraining Models
Unit 4: Alerting and Performance Optimization
Setting Up Alerts
Analyzing Performance Data
Optimizing Model Performance
Model Versioning
Data Governance and Security in MLOps Workflows
Unit 1: Data Governance in MLOps
Data Governance Intro
Data Lineage Tracking
Data Cataloging
Delta Lake Governance
Data Quality Checks
Unit 2: Access Control and Data Encryption
Access Control Overview
Workspace Access Control
Table Access Control
Data Encryption Basics
Encryption at Rest
Encryption in Transit
Unit 3: Auditing and Logging
Auditing and Logging
Analyzing Audit Logs
Monitoring Access Patterns