# System One Primitives [[System One Models]] like [[Jev]] don't take a prompt and return text. You send a **state** (the content to judge) and a set of typed **questions**, and each question uses one of three primitives: **Choice**, **Score** or **Noul**. That's the entire interface. Once you understand these three shapes (and their quirks), you understand what a [[Decision Models (DMs)|decision model]] can and can't do for you. [[Diogo Almeida]] maps them to programming constructs, which is the best mental model I've found: **Choice is a `switch`, Noul is an `if`, Score is a sort or a threshold**. ## Anatomy of a request - **State**: a string, a JSON object or an array. You can point at parts of it from a question with a backticked path, e.g. `` `ticket.messages[0].text` `` - **Questions**: a map from your own ID to a question with a `type`, `instructions` (what to judge) and, depending on the type, `criteria` (the answer space and what it means) - **Question IDs are never sent to the model.** Write the full question in `instructions` even when the ID looks obvious. It's a classic mistake - **Two guarantees.** *Constrained*: the answer is always a distribution over what you declared, nothing parsed from prose. *Independent*: one question never sees another's answer, so adding or removing questions doesn't change the others - **Many questions per request are cheap.** The state is read once and every question runs in parallel against it. In the parallel questions cookbook, 13 questions in one call vs 13 separate calls came out **12.2x cheaper and 10.0x faster, with identical answers** (another docs page says 11.5x and 9.6x for the same experiment) Limits for `jev-1.13`: 64k tokens per request (state plus all questions), 32k for the state plus the longest question, text only (no images or audio), best in English. Price: $0.042 per million input tokens, output free. ## Choice: "which of these?" A probability distribution over options you provide (up to 255, i.e. 2^8 - 1). It returns `choice` (the argmax), `probabilities` (summing to 1) and `confidence`. Options can carry descriptions; both name and description are sent to the model. Use it for unordered categories: department, document type, language. Give the full list rather than a shortlist (each option costs a few tokens), and add `other` or `none of the above` if the list may not cover every input. The docs' ambiguous ticket example (late delivery + wrong size + double charge) shows why the full distribution matters: `department` = returns 0.61, billing 0.35, confidence 0.42. The code sends the ticket to returns *and* copies billing because 0.35 is above a 0.25 threshold. You can't do that with a label. ## Score: "which level?" An ordered rubric of 2 to 10 levels, low to high, each described in words. It returns the **expected value** of the level distribution: score = Σ level × P(level). With P = 0.00 / 0.57 / 0.43 on levels 0 / 1 / 2, the score is **1.43**. So a score is a position between levels, not a class. The gotchas are where the learning is: - **Different distributions give the same score.** 1.0 can mean "all on level 1" or "half on 0, half on 2". Always read `probabilities` and `confidence` next to `score` - **Each level is judged on its own.** The model never sees level numbers or neighbors. Numeric-only levels (`["0","1","2"]` with "rate 0-2") on a misaligned-button bug gave 0.55 at confidence 0.33; descriptive levels gave 0.0 at confidence 1.0 - **Describe situations, not degrees.** "Broken feature, but a workaround exists" beats "moderately severe". One dimension per Score; give a rare extreme its own top level - **Examples help only when they resemble real inputs.** On a Safari bug report, plain levels gave 1.43 at confidence 0.35; adding a matching example ("fails in one browser but works in another") gave 1.03 at 0.96; an unrelated example changed nothing - **Normalize before combining.** Divide by the top level (`len(criteria) - 1`) to get 0-1, then weights mean what they say ([[Composite Scoring]]) - **Don't interpolate magnitudes.** Thresholding the expectation is fine; reconstructing "exactly 2.7 days of delay" from it isn't ## Noul: "is this true?" A single number in [0, 1]: the probability that the answer is yes. There's no separate confidence field, because one number fully describes a two-outcome distribution (near 0.5 means unsure). **The name.** The docs never explain it, and the API calls it `noul`. On the Latent Space podcast, Almeida says "Noulli" and explains it comes from **Bernoulli**, as in a Bernoulli probability (a yes/no coin with probability p). So: Noul in the API, Noulli in conversation, Bernoulli at the root. "Is the customer asking for a human agent?": | Message | noul | |---|---| | "Thanks, that fixed it!" | 0.02 | | "How do I reset my password?" | 0.07 | | "Are you a bot?" | 0.40 | | "Is there any way to speak to someone about my invoice?" | 0.84 | | "I have asked three times now. Can I please just talk to a real person?" | 0.99 | A Noul is a probability, **not a degree**. "Is the candidate strong in Python?" gave 0.03 / 0.14 / 0.81 / 0.92 for no Python / occasional scripts / two years daily / eight years, while a 4-level Score gave 0.0 / 1.0 / 2.05 / 2.89. A Noul of 0.5 can mean "medium" or "unclear", and you can't tell which. Use a Score for "how much". Writing rules: one condition per Noul (split "angry AND wants a refund"), phrase it so yes is the high value (not "Is it free of PII?"), and pick thresholds from the cost of error: 0.5 when both errors cost the same, higher when a false yes is expensive (paging someone), lower when a missed yes is (safety), and a human for the middle. ## Confidence Choice and Score answers include a `confidence`. From the docs' examples, it's the top probability linearly rescaled between "uniform" and "certain": `confidence = (n × p_max − 1) / (n − 1)`, clamped to [0, 1], where n is the number of options or levels. Check: p_max 0.61 over 3 options gives (1.83 − 1) / 2 = 0.42, the value in the ticket example above. The docs call it a "solid default" and point out that you get the full `probabilities`, so you can compute your own (margin, entropy). Two warnings: - **Confidence describes the shape of the answer, not whether it's correct.** It only becomes a correctness signal if the model is calibrated on your data ([[AI Model Calibration]]) - One cookbook claims confidence separates "0.45 vs 0.44" from "0.45 with the rest scattered". With the formula above, both give the same confidence (it only looks at p_max). The docs contradict themselves here; if the margin matters to you, compute it from `probabilities` ## Which primitive? | Your question | Primitive | |---|---| | Which category, no natural order | Choice | | How much, how severe, which level | Score | | Is X true, does X exist, should this pass | Noul | | Which of these, and is any of them right at all | Choice + one Noul per option ([[Relative vs Absolute AI Judgments]]) | | Extract a value | Choice over pre-extracted candidates ([[Candidate-Then-Select Extraction]]) | And never assume the three forms agree with each other. The same refund question gave 0.22 as a Noul and 0.01 as a yes/no Choice; "refund?" and "not a refund?" summed to 1.19. Don't carry a threshold from one form to another. ## My take The design choice I like most is that the output is *boring*: a number or a label, typed by construction. That's what makes it testable. The part people underestimate is how much the wording of `criteria` matters (numeric levels vs situational levels was the difference between a coin flip and certainty). Writing good questions is the new prompt engineering, just with a much tighter feedback loop. ## References - [Primitives (TypeSafe docs)](https://docs.typesafe.ai/primitives) - [Choice (TypeSafe docs)](https://docs.typesafe.ai/primitives/choice) - [Score (TypeSafe docs)](https://docs.typesafe.ai/primitives/score) - [Noul (TypeSafe docs)](https://docs.typesafe.ai/primitives/noul) - [Confidence (TypeSafe docs)](https://docs.typesafe.ai/confidence) - [Parallel questions cookbook (TypeSafe docs)](https://docs.typesafe.ai/cookbooks/parallel_questions) - [Jev: System One models for Prod, not God, with Diogo Almeida (Latent Space)](https://www.latent.space/p/jev) ## Related - [[Jev]] - [[System One Models]] - [[Decision Models (DMs)]] - [[Atomic Question Decomposition]] - [[Relative vs Absolute AI Judgments]] - [[Composite Scoring]] - [[Confidence-Gated Routing]] - [[Speculative Fan-Out]] - [[AI Model Calibration]] - [[Candidate-Then-Select Extraction]] - [[Binary Classification]] - [[LLM Structured Outputs]]