# AI Fine-Tuning The process of further training a pre-trained model on task-specific data to adapt its behavior. Captures deep behavioral changes that persist in the model's weights. Types of fine-tuning: - **Full fine-tuning**: updates all model parameters. Most expressive but most expensive and requires the most data - **Parameter-efficient fine-tuning**: methods like [[Low Rank Adapter (LoRA)]], QLoRA, and adapters that update only a small subset of parameters. Dramatically cheaper with comparable results for many tasks - **Instruction tuning**: training on instruction-response pairs to make the model follow directions. [[Reinforcement Learning From Human Feedback (RLHF)]] is often applied on top of this Trade-off: fine-tuning captures deep behavioral changes but is expensive and can cause catastrophic forgetting (the model loses previously learned capabilities). Compare with in-context learning via prompts, which is cheaper and more flexible but less persistent and limited by the context window. [[Knowledge Distillation]] is a related technique where a smaller model is trained to mimic a larger one, effectively "fine-tuning" a compact model to replicate the behavior of a frontier model. ## References ## Related - [[Large Language Models (LLMs)]] - [[Low Rank Adapter (LoRA)]] - [[Knowledge Distillation]] - [[Reinforcement Learning From Human Feedback (RLHF)]]