GenAI for Seasoned PMs: LLM Training & Evaluation Fundamentals
A concise course equipping experienced Product Managers with the foundational knowledge of LLM training and evaluation to excel in the age of Generative AI.
...
Share
GenAI and LLM Landscape for Product Managers
Unit 1: Introduction to Generative AI
Welcome, PM!
What is Generative AI?
GenAI Use Cases
Unit 2: LLMs: The Power Behind GenAI
What are LLMs?
LLM Architecture
LLM Components
Unit 3: LLMs for Product Managers
LLM Types
LLM Applications
Ethical Considerations
LLM Limitations
LLM Training Data Fundamentals
Unit 1: The Primacy of Data in LLM Training
Why Data is King
Data's Impact on LLMs
Unit 2: Sourcing and Preparing Training Data
Data Sourcing Strategies
Data Cleaning Techniques
Data Prep & Formatting
Unit 3: Data Bias and Mitigation
Identifying Data Bias
Mitigating Data Bias
Unit 4: Data Augmentation and Generalization
Data Augmentation Intro
Applying Augmentation
Generalization Strategies
LLM Training and Fine-Tuning Essentials
Unit 1: LLM Training Methodologies
Training Methodologies
Pre-training Explained
Fine-Tuning Deep Dive
Unit 2: Transfer Learning and Hyperparameters
Transfer Learning
Hyperparameter Basics
Hyperparameter Tuning
Unit 3: Optimization and Monitoring
Optimization Algorithms
Monitoring Training
Evaluating Training
Putting It All Together
LLM Evaluation Metrics and Case Study
Unit 1: LLM Evaluation Metrics
Intro to LLM Evaluation
Perplexity Explained
BLEU: Translation Metric
ROUGE: Summarization Metric
Beyond BLEU & ROUGE
Unit 2: LLM Evaluation Methods and Weaknesses
Human Evaluation Methods
Diagnosing Model Weaknesses
Prompt Engineering Impact
Unit 3: Case Study and Practical Application
Case Study: Chatbot Eval
Scenario: GenAI for PMs