# Typicality Bias Typicality bias is our tendency to prefer what feels familiar. Show people two answers of equal quality, and they'll pick the more typical one more often than chance. That's a well-known effect in cognitive psychology (see [[Cognitive biases]]), and it becomes a real problem the moment we use human preferences to train AI models. The term, applied to LLMs, comes from Jiayi Zhang, Simon Yu, Derek Chong, Anthony Sicilia, Michael Tomz, Christopher Manning and Weiyan Shi (Stanford and Northeastern), in *Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity* (arXiv:2510.01171, October 2025). Their claim: the blandness of chatbots isn't only an algorithm problem. It's baked into the data. ## Why it causes mode collapse The standard story about [[Mode Collapse|mode collapse]] blames the training algorithm: [[Reinforcement Learning From Human Feedback (RLHF)|RLHF]] with an imperfect reward model, over-optimization, the KL penalty. Zhang et al. point somewhere else. Preference data is made by humans, and humans (the annotators) systematically favor familiar text. The paper grounds this in the mere-exposure effect, the availability heuristic, processing fluency (easy-to-read text feels more true) and schema congruity (familiar content gets less scrutiny). They formalize it with one extra term. The reward people actually give looks like this: r(x, y) = r_true(x, y) + α · log π_ref(y | x) In plain words: the reward is the *real* quality of the answer, plus a bonus for how typical it is under the base model. Plug that into the RLHF objective (β is the strength of the KL penalty that keeps the model close to its base) and the optimal policy becomes π*(y | x) ∝ π_ref(y | x)^γ · exp(r_true(x, y) / β), with γ = 1 + α/β. That exponent is greater than 1 whenever α > 0. Raising a distribution to a power above 1 *sharpens* it: common answers get more common, rare ones fade away. And α isn't hypothetical. On the HelpSteer preference dataset, comparing pairs of answers with the same correctness, they estimated α ≈ 0.57 and 0.65 (using Llama 3.1 405B and GLM-4.5 as the base model), with p < 10⁻¹⁴. The consequence I find the most important: this happens **even with a perfect reward model and a perfect optimizer**. You can't fix it with better RL. The bias is in the labels. ## Ask for a distribution instead of an answer The fix, which the authors call Verbalized Sampling, is a prompting trick, and it's almost silly how simple it is. Instead of asking for one answer, ask the model to list several answers *along with their probabilities*, then sample from that list. Why does that work? When you ask for one joke about coffee, the post-trained model gives you the mode: the single most typical joke (the same one every time). When you ask for five jokes with their probabilities, the most typical response to *that* request is a spread-out distribution, and the model is good at producing one. The diversity from pretraining is still in there. You're just asking a question whose typical answer is diverse. Results from the paper: - **1.6 to 2.1x more diversity** in creative writing (poems, stories, jokes) compared to direct prompting - Gains also in dialogue simulation, open-ended QA and synthetic data generation - No loss of factual accuracy or safety - More capable models benefit more from it There are variants: VS-Standard (k answers with probabilities in one call), VS-CoT (reason first, then produce the distribution) and VS-Multi (spread the answers over several turns). ## A prompt you can use today This is the prompt from the authors' repository. Paste it before your request: ```text <instructions> Generate 5 responses to the user query, each within a separate <response> tag. Each <response> must include a <text> and a numeric <probability>. Please sample at random from the tails of the distribution, such that the probability of each response is less than 0.10. </instructions> Give me 5 titles for an article about why my notes app became a graveyard. ``` The "less than 0.10" part pushes the model away from the obvious answers and into the tails. Drop it (or raise it) when you want the *likely* answers with an honest spread instead. You can also put the same instructions in a system prompt so every answer comes back as a small distribution. Where I'd use it: titles, hooks, [[Brainstorming]], naming, story ideas, synthetic test data. Anywhere "the most typical answer" is exactly what you don't want. ## Why it matters beyond creative writing - **Sycophancy has the same root.** Preference data rewards what people like to read, and people like to read agreement (see [[AI Sycophancy]]). Typicality bias and sycophancy are two faces of "optimize for what raters prefer" - **Calibration too.** The sharpening that kills diversity also pushes probabilities toward overconfidence (see [[AI Model Calibration]]). Same mechanism, seen from another angle - **It's part of the case for decision models.** [[Jev]]'s pitch is that you shouldn't train a model that makes decisions on human preferences at all. A classifier whose distribution is sharpened toward the typical label will under-predict the rare classes, which are often the ones you care about. TypeSafe trains with [[Reinforcement Learning for Calibrated Decisions (RLCD)|RLCD]] instead ## My take This paper changed how I think about the "AI voice". The sameness isn't a lack of capability; the model *knows* more varied answers and was trained to hide them because we, the humans, rewarded the familiar. I find that humbling (it's our bias, not the machine's). And the practical lesson is great news: you don't need a different model to get variety, you need a different question. Ask for a distribution, not an answer. ## References - [Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity (Zhang et al., arXiv:2510.01171)](https://arxiv.org/abs/2510.01171) - [Verbalized Sampling code and prompts (GitHub, CHATS-lab)](https://github.com/CHATS-lab/verbalized-sampling) - [Understanding the Effects of RLHF on LLM Generalisation and Diversity (Kirk et al., arXiv:2310.06452)](https://arxiv.org/abs/2310.06452) ## Related - [[Mode Collapse]] - [[Reinforcement Learning From Human Feedback (RLHF)]] - [[AI Sycophancy]] - [[AI Post-Training]] - [[AI Model Calibration]] - [[Cognitive biases]] - [[Prompt Engineering]] - [[Brainstorming]] - [[AI Temperature]] - [[The impact of AI focus on creative diversity]]