“In AI development, primitives are the fundamental, low-level building blocks – such as prompt routing, vector memory, tool calling, and evaluation steps – used to construct modular and reliable AI workflows. Rather than relying on rigid, monolithic frameworks that obscure the underlying logic, developers treat these modular components as atomic units of capability that can be custom-stitched together.” – AI primitive – Artificial intelligence
Reliable AI applications are assembled from decisions that can be inspected, tested, replaced, and governed independently. A model may generate language, but production behaviour depends on the surrounding mechanisms that select context, invoke services, preserve state, constrain actions, and check results. Treating these mechanisms as modular units changes development from configuring one opaque application into designing a system whose capabilities and failure modes can be examined piece by piece.
An AI primitive is therefore a small, reusable unit of capability or control in an artificial intelligence workflow. It need not be a single software function, and it is not necessarily tied to one vendor or model. A retrieval operation, a prompt selector, a structured-output validator, a tool interface, a memory store, a human approval gate, and an evaluation routine can each serve as a primitive when it has a clear purpose and boundary. The useful test is practical: can the component be understood, measured, substituted, and composed with others without rewriting the whole application?
This approach distinguishes a capability from a framework. A framework supplies conventions, abstractions, integrations, and often a ready-made execution model. It can accelerate prototyping, but it may also conceal control flow and make debugging dependent on framework-specific behaviour. Primitives expose more of the underlying logic. Developers can use a simple sequence of model calls and ordinary program code when that is sufficient, then add routing, retrieval, tools, or longer-running loops only where the task requires them. Research and engineering guidance similarly distinguishes predictable workflows, in which code determines the path, from agents, in which a model dynamically selects actions.1
How the building blocks work together
Prompt routing is a decision layer that directs an input to an appropriate instruction set, model, tool chain, or level of scrutiny. A classifier might send a billing query to a narrow workflow and a complex research request to a more capable one. Routing can reduce cost and latency, but it introduces its own risks: ambiguous requests may be assigned to the wrong path, and a routing model can become a hidden source of bias or inconsistency. Its quality must therefore be assessed separately from the quality of the downstream response.
Vector memory is another common primitive. Documents, records, or prior interactions are converted into numerical representations and stored so that semantically related material can be retrieved later. Retrieval can improve factual grounding without retraining the main model, yet similarity is not the same as relevance. A system may retrieve outdated, duplicated, sensitive, or misleading material. Effective designs combine semantic search with metadata filters, access controls, freshness rules, and explicit handling for cases in which no trustworthy source is found.
Tool calling connects a model to external operations such as databases, calculators, search services, code execution, or transactional systems. The interface should specify the permitted inputs, outputs, permissions, error states, and side effects. Documentation is not cosmetic: the model uses the available description to decide whether and how to invoke a tool, so vague names and broad schemas can produce unreliable actions. Anthropic recommends testing tool use systematically and measuring not only task success but also tool errors, runtime, call volume, and token consumption.1
Memory, tools, and retrieval also create a boundary between language generation and real-world consequence. A generated answer can be corrected in conversation; an unauthorised database update or financial transaction may be difficult to reverse. Strong systems separate read operations from write operations, limit credentials, validate arguments in deterministic code, and place approval gates around consequential actions. Sandboxed testing and explicit guardrails are especially important when an agent can alter files, call external services, or continue through multiple steps.2
Specification, composition, and evaluation
Composition is valuable because different tasks need different arrangements. A sequential workflow may retrieve evidence, draft an answer, verify citations, and format the result. Parallel branches may ask several specialised components to inspect the same input before an aggregator reconciles them. An evaluator-optimiser pattern can generate an output, score it against defined criteria, and revise it. These patterns are not automatically superior to a single model call. Added steps increase latency, cost, state-management complexity, and the number of places where errors can accumulate. The appropriate architecture is the simplest one that meets the required quality and control level.2
Evaluation is itself a primitive rather than a final ceremonial check. A useful evaluation defines realistic tasks, observable success criteria, representative edge cases, and a repeatable grading method. Code-based tests are suitable for exact formats and executable outcomes; model-based graders can assess open-ended quality; human review remains important where context, fairness, safety, or professional judgement matter. Anthropic describes agent evaluation as requiring an environment in which the system can use its tools and produce an outcome that can be checked, rather than merely generating an attractive response.3
Formally, a workflow can be viewed as a composition of functions, where each component transforms a state into a new state: s_{k+1}=f_k(s_k,x_k). Here s_k is the system state at step k, x_k is the relevant input or observation, and f_k is a primitive such as retrieval, routing, generation, validation, or tool execution. If the final outcome is scored by an evaluation function J(y,y^*), where y is the produced result and y^* is the target outcome, improvement requires measuring how changes to individual f_k affect quality, cost, latency, and risk. The notation does not make the system reliable by itself; it clarifies where reliability can be tested.
Competing design philosophies
One school favours high-level platforms and opinionated frameworks. Their advantage is speed: standard connectors, tracing, memory abstractions, retries, and agent loops can be assembled quickly, allowing teams to validate a use case before building infrastructure. Another school favours minimal orchestration and explicit code. Its priorities are transparency, portability, predictable control flow, and freedom from abstractions that may become difficult to replace. A third position treats the choice as conditional. High-level components are useful when they reduce undifferentiated engineering work, but core decisions should remain visible enough to audit and test.
The main tension is not frameworks versus no frameworks. It is convenience versus control. A reusable primitive can reduce duplication, yet excessive fragmentation can create an architecture that is hard to understand and expensive to observe. Stable interfaces are essential: each component should state its contract, assumptions, failure behaviour, security requirements, and performance expectations. Versioning matters as well, because changes to a prompt, embedding model, retrieval index, tool schema, or evaluator can alter downstream behaviour even when application code remains unchanged.
Governance must follow the same modular logic. NIST’s AI Risk Management Framework organises practical risk work around governing, mapping, measuring, and managing risks, and recommends testing before deployment and during operation.4 Its generative AI profile adds emphasis on ongoing monitoring, documented evaluation, incident response, and review of risk controls.5 These recommendations reinforce a central implication of primitive-based design: quality, safety, provenance, and access control should be observable at the points where decisions and actions occur, not inferred only from the final text.
The concept remains important because model capability alone does not determine system performance. Organisations need to replace models, update knowledge, introduce new tools, control costs, and adapt to changing regulations without rebuilding every application. Modular primitives make those changes technically possible and analytically legible. They also expose a less comfortable truth: composability does not remove risk. It distributes risk across interfaces, state transitions, permissions, data sources, and evaluation gaps. The strongest AI systems therefore combine small, explicit building blocks with disciplined testing, constrained authority, production monitoring, and human oversight where consequences warrant it.
References
1. Writing effective tools for AI agents-using … – 2025-09-11 – https://www.anthropic.com/engineering/writing-tools-for-agents
2. Building Effective AI Agents – 2024-12-19 – https://www.anthropic.com/engineering/building-effective-agents
3. www.nist.gov · itl · ai-risk-management-frameworkAI Risk Management Framework | NIST – 2021-07-12 – https://www.nist.gov/itl/ai-risk-management-framework
4. Building Effective AI Agents – Anthropic – https://resources.anthropic.com/building-effective-ai-agents
5. Artificial Intelligence Risk Management Framework: Generative … – 2024-07-26 – https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence
6. [PDF] Building Effective AI Agents: Architecture Patterns and … – Anthropic – https://resources.anthropic.com/hubfs/Building%20Effective%20AI%20Agents-%20Architecture%20Patterns%20and%20Implementation%20Frameworks.pdf
7. [PDF] NIST.AI.600-1.GenAI-Profile.ipd.pdf – https://airc.nist.gov/docs/NIST.AI.600-1.GenAI-Profile.ipd.pdf
8. Artificial Intelligence Risk Management Framework (AI RMF 1 – https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf
9. Comment on NIST RMF GenAI Companion – https://downloads.regulations.gov/NIST-2024-0001-0075/attachment_2.pdf
10. Demystifying evals for AI agents – 2026-01-09 – https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents
11. Building Effective AI Agents – Anthropic | PDF | Artificial Intelligence – 2025-05-28 – https://www.scribd.com/document/878600615/Building-Effective-AI-Agents-Anthropic
12. nvlpubs.nist.gov · nistpubs · aiArtificial Intelligence Risk Management Framework: Generative … – https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf?_sp=f87900c4-bd3e-4d78-895f-88a2e572f7e2
13. AI Risk Management Framework – NIST Computer Security … – https://csrc.nist.gov/csrc/media/Presentations/2022/ai-risk-management-framework/Day%201%20-%201115am%20Tabassi.pdf
14. Artificial Intelligence Risk Management Framework (AI … – 2023-01-26 – https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-ai-rmf-10
15. Publications – 2024-07-26 – https://www.nist.gov/publications/search_by_author/1838481
