# Machine Learning (ML)
Machine Learning is a subset of [[Artificial Intelligence (AI)]] where systems learn patterns from data rather than being explicitly programmed. Instead of writing rules to solve problems, ML algorithms discover rules by analyzing examples. The term was coined by [[Arthur Samuel]] in 1959 while developing a checkers-playing program at IBM that improved through self-play.
ML has become foundational to modern technology: recommendation systems (Netflix, Spotify), search engines, fraud detection, medical diagnosis, autonomous vehicles, and language models. The field experienced dramatic growth after 2012 when [[Deep Learning]] achieved breakthroughs in image recognition, followed by advances in natural language processing, leading to today's generative AI systems.
## Types of Machine Learning
| Type | Description | Examples |
|------|-------------|----------|
| **Supervised** | Learn from labeled data | Classification, regression |
| **Unsupervised** | Find patterns in unlabeled data | Clustering, dimensionality reduction |
| **Reinforcement** | Learn from rewards/penalties | Game playing, robotics |
| **Self-supervised** | Create labels from data itself | GPT pretraining, BERT |
## Supervised Learning
Learn mapping from inputs to outputs:
```
Training Data: Input (X) → Output (Y)
[features] [label]
Model learns: f(X) ≈ Y
Prediction: New X → Predicted Y
```
**Tasks**:
- **Classification**: Predict category (spam/not spam)
- **Regression**: Predict continuous value (price, temperature)
## Common Algorithms
| Algorithm | Type | Use Case |
|-----------|------|----------|
| **Linear Regression** | Supervised | Predict continuous values |
| **Logistic Regression** | Supervised | Binary classification |
| **Decision Trees** | Supervised | Classification, feature importance |
| **Random Forest** | Supervised | Ensemble of trees |
| **SVM** | Supervised | Classification with margins |
| **K-Means** | Unsupervised | Clustering |
| **PCA** | Unsupervised | Dimensionality reduction |
| **Neural Networks** | Supervised/Other | Complex pattern recognition |
## ML Workflow
```
1. Define Problem
↓
2. Collect Data
↓
3. Prepare Data (clean, transform, split)
↓
4. Choose Model
↓
5. Train Model
↓
6. Evaluate (validation set)
↓
7. Tune Hyperparameters
↓
8. Final Evaluation (test set)
↓
9. Deploy
```
## Key Concepts
| Concept | Description |
|---------|-------------|
| **Training set** | Data used to train model |
| **Validation set** | Data for tuning hyperparameters |
| **Test set** | Final evaluation data |
| **Overfitting** | Model memorizes training data, fails on new data |
| **Underfitting** | Model too simple, misses patterns |
| **Regularization** | Prevent overfitting (L1, L2, dropout) |
| **Cross-validation** | Robust evaluation using data splits |
| **Feature engineering** | Creating useful input features |
## Bias-Variance Tradeoff
```
Total Error = Bias² + Variance + Irreducible Error
High Bias (Underfitting): High Variance (Overfitting):
Simple model, misses patterns Complex model, fits noise
```
## ML vs Traditional Programming
| Traditional | Machine Learning |
|-------------|------------------|
| Rules + Data → Output | Data + Output → Rules |
| Programmer defines logic | Algorithm discovers logic |
| Explicit instructions | Learned patterns |
## References
- https://en.wikipedia.org/wiki/Machine_learning
- Bishop. *Pattern Recognition and Machine Learning*
- Hastie et al. *Elements of Statistical Learning*
## Related
- [[Artificial Intelligence (AI)]]
- [[Deep Learning]]
- [[Neural Networks (NNs)]]
- [[Supervised Learning (SL)]]
- [[Unsupervised Learning]]
- [[Reinforcement Learning (RL)]]
- [[Data Science]]
- [[WebMachineLearning]]
- [[On-Device Machine Learning]]
- [[WebNN API]]