# Phi LLMs Phi is a family of small language models (SLMs) created by Microsoft Research. The core thesis is that data quality matters more than model size: small models (1.3B-15B parameters) trained on high-quality curated and synthetic data can match or exceed models 10-25x their size. All models are released under the MIT license. Led by Sebastien Bubeck (VP of Generative AI Research), Yuanzhi Li, Marah Abdin, and others from the Machine Learning Foundations team. ## Model Timeline | Model | Date | Parameters | Focus | |-------|------|-----------|-------| | Phi-1 | Jun 2023 | 1.3B | Python code generation | | Phi-1.5 | Sep 2023 | 1.3B | General reasoning | | Phi-2 | Dec 2023 | 2.7B | General-purpose SLM | | Phi-3-mini | Apr 2024 | 3.8B | On-device capable (phone) | | Phi-3-small | Apr 2024 | 7B | Balanced performance | | Phi-3-medium | Apr 2024 | 14B | Higher capability | | Phi-3.5-MoE | Aug 2024 | 6.6B active | Mixture-of-Experts | | Phi-3.5-Vision | Aug 2024 | - | Multimodal (text + images) | | Phi-4 | Dec 2024 | 14B | Complex reasoning, math, STEM | | Phi-4-mini | Feb 2025 | 3.8B | Compact reasoning/coding | | Phi-4-multimodal | Feb 2025 | 5.6B | Text, audio, image input | | Phi-4-reasoning | May 2025 | 14B | STEM reasoning (beats DeepSeek-R1 70B) | | Phi-4-reasoning-vision | Mar 2026 | 15B | Multimodal reasoning (latest) | ## Training Approach 1. **Curated web data**: filtered for "textbook quality" relevance and clarity 2. **Synthetic data generation**: using larger models to generate training exercises and reasoning chains 3. **Curriculum learning**: progressive training from simpler to more complex tasks 4. **Focused fine-tuning**: reasoning-specific variants fine-tuned on millions of STEM/coding problems 5. **Reinforcement learning**: "plus" variants add an RL stage for further reasoning improvement The original insight came from the paper "Textbooks Are All You Need" (Phi-1), which showed that training on 7B tokens of textbook-quality data produced a 1.3B model that surpassed GPT-3.5 on coding benchmarks. ## Key Characteristics - All models range from 1.3B to 15B parameters - Designed for on-device deployment without cloud connectivity - Consistently match or beat models 5-25x their size on reasoning, math, and coding benchmarks - Open source under MIT license ## References - https://azure.microsoft.com/en-us/products/phi - https://github.com/microsoft/PhiCookBook - https://arxiv.org/abs/2306.11644 - https://huggingface.co/microsoft/phi-4 ## Related - [[Large Language Models (LLMs)]] - [[Transformers]] - [[Windows Copilot Runtime]] - [[Small Language Models (SLMs)]] - [[On-Device Machine Learning]]