An AI model is a versioned artifact that maps inputs to outputs under a training and serving contract—not a synonym for artificial intelligence, and not a stand-in for every deep architecture textbook. This guide owns cross-model taxonomy, capability dimensions, evaluation for selection, open-versus-closed trade-offs, lifecycle operations, and enterprise selection scorecards. Vendor examples include OpenAI and Anthropic. Internals of language modeling stay in large language models. Cross-modality generative product patterns stay in generative AI. Representation-learning systems context sits in deep learning; unit-level nets in neural networks. Cluster serving economics deepen in AI infrastructure.
Buyers and platform architects need a way to compare models inside a task family without fake universal leaderboards. That is the job of this page.
What “a model” means as an artifact
A model artifact typically includes weights (or a hosted endpoint identity), a tokenizer or preprocessor, configuration, and a documented input/output schema. Treat the whole bundle as the unit of change. Swapping weights while silently changing chat templates or normalization flags is a new model—even if the marketing name is unchanged.
Models are decision tools: they emit scores, classes, embeddings, tokens, pixels, or structured tool calls. The useful question is “what decision does this artifact improve?” not “what is the biggest parameter count?” Capability claims without decision mapping produce shelfware.
Version pins, model cards, eval suite hashes, and license identifiers belong in a registry beside the artifact. Without registry discipline, “the production model” becomes folklore.
Taxonomy by function and modality
Classify models first by function, then by modality. Functional families include: predictive (classify, regress, rank, detect), generative (synthesize text, image, audio, code), embedding (map items to vectors), reranking (reorder candidates), and policy/safety classifiers. Products often combine several functions; taxonomy still forces clarity about which function you are buying.
Modality axes—text, image, audio, video, tabular, multimodal—constrain architecture choices and data contracts. A vision detector and a text classifier are both “predictive,” yet they do not share eval suites or failure modes. Compare inside a cell of the function×modality grid.
| Function | Typical outputs | Selection focus |
|---|---|---|
| Predictive | Labels, scores, boxes | Calibration, cost of errors |
| Generative | Tokens, pixels, audio | Controllability, abuse, fidelity |
| Embedding | Vectors | Retrieval recall, metric parity |
| Reranker | Ordered lists | nDCG lift vs latency |
| Safety/policy | Allow/block/scores | False block vs miss rates |
This taxonomy is intentionally shallower than modality deep dives. When you need tokenizer behavior or diffusion stacks, leave this page for the specialized guides.
Foundation versus specialist models
Foundation models are broad pretraining artifacts adapted via prompting, adapters, or fine-tuning. Specialist models are trained or heavily adapted for a narrow distribution—fraud scores, medical imaging triage, industrial OCR. Foundations win on interface leverage and transfer; specialists win when the decision metric is sharp and the domain is far from web pretraining.
Do not assume a frontier generative model dominates every predictive task. Classical machine learning and compact specialist nets suited to edge AI often beat general chat models on calibrated tabular risk, latency, or auditability. Selection is portfolio design: route easy or structured tasks to specialists; reserve foundations for open-ended interface work.
Multimodal foundations blur modality boundaries. Still score them on the slices you ship—not on demo reels that mix capabilities you will never expose.
Capability dimensions that matter
Scorecards should include: task quality on private suites; latency percentiles; throughput under concurrency; context or input size limits; controllability (schemas, constraints); multilingual or domain coverage; tool-calling reliability where relevant; refusal/overrefusal behavior for user-facing systems; and operational properties (versioning, residency, retention).
Parameter count and public leaderboard rank are weak proxies. Private golden sets beat screenshots. For embeddings, measure recall@k on your corpus; for detectors, measure the operating point that matches your false-positive budget; for generators, measure task success and safety jointly.
Cost is a capability constraint. Dollars per million tokens or per image interact with UX verbosity and retry policy. Model choice without cost ceilings is incomplete architecture.
Evaluation frameworks that match decisions
Align eval to the decision: ranking metrics for search stages, calibration for risk scores, faithfulness for grounded answers (owned end-to-end by RAG), execution tests for code, human preference samples for open-ended chat. A single “accuracy” number across families is marketing, not engineering.
Offline suites need versioned prompts, fixed seeds where possible, and stratified slices (language, length, rare classes). Online eval and shadow traffic catch distribution shift that offline suites miss. When providers update endpoints, re-run suites before raising traffic.
Public benchmarks remain useful for rough screening and research comparison. They are contaminated, gameable, and weakly predictive of your domain. Use them as gates to a shortlist, not as purchase evidence.
Hosting: API, VPC, and on-prem weights
Hosting choices change control, latency, residency, and ops burden. Managed APIs optimize iteration speed and frequent capability upgrades. VPC or private endpoints improve network isolation while still outsourcing weight ops. Self-hosted open weights maximize control and customization at the cost of serving expertise, patching, and capacity planning—terrain that overlaps AI infrastructure without replacing that guide’s stack depth.
Hybrid portfolios are common: API for peak generative quality, self-host for high-volume embeddings or sensitive predictive models. Document which surfaces may call which hosting class.
Hosting is not model quality. A weak model on premium GPUs remains weak. A strong model with broken chat templates or quantization regressions fails in production. Evaluate the serving configuration you will actually run.
Open weights versus closed API models
Open weights (under various licenses) enable inspection, offline air gaps, and deep customization. Closed APIs optimize convenience and often lead capability curves. Neither is automatically safer or cheaper. License compatibility, training-data disclosures, and dangerous-capability evals still apply to open weights. Retention policies, subprocessors, and version pins still apply to APIs.
Exit drills matter: can you re-score golden suites on an alternate model within a defined window? Concentration risk is a selection criterion, not an afterthought.
“Open” does not mean “finished product.” You still own eval, filters, serving, and updates.
Licensing and usage constraints
Track license terms for weights, training-on-prompts defaults for APIs, output ownership, redistribution of derivatives, and geographic restrictions. Legal review is not optional for enterprise adoption; this page only insists that license IDs live next to model IDs in the registry.
Fine-tuning and distillation can create derivative artifacts with distinct license obligations. Record lineage when adapters or distilled students ship.
Selection scorecards for enterprises
Build a scorecard template per task family: must-have capabilities, hard constraints (residency, latency SLO, max cost), eval suite links, hosting options, license status, security review status, and rollback plan. Weight dimensions explicitly so “wow demos” cannot override hard constraints.
Run bake-offs on identical suites and identical preprocessing. Changing chunking, prompts, and models simultaneously invalidates comparisons—especially in retrieval systems that use embeddings.
Include negative selection: when rules, classical ML, or retrieval-only answers beat a generative model. Non-selection is a valid scorecard outcome.
Lifecycle: upgrade, deprecate, and regress
Models age. Providers deprecate endpoints; open checkpoints get superseded; adapters drift relative to new bases. Treat upgrades like dependency bumps: changelog review, suite diffs, canary, rollback. Deprecation needs communication SLAs to product owners.
Regression classes differ by family: formatting breaks, tool-schema breaks, calibration drift, retrieval recall drops, safety miss rates. Route pages to owners who understand the family—not a single “AI quality” queue.
Pin versions for regulated or high-traffic surfaces. Floating “latest” tags are acceptable only where suite automation and auto-rollback exist.
How this guide differs from LLM, GenAI, DL, and NN deep dives
LLMs own tokenization, pretrain/align stacks, and LM inference knobs. Generative AI owns modality maps and controllability at product altitude. Deep learning owns train/serve representation systems. Neural networks own units, losses, and inductive biases. This guide owns the selection layer that sits above those specialties: which artifact class, which scorecard, which hosting posture, which lifecycle controls.
If a paragraph starts explaining attention math or diffusion samplers in depth, it belongs elsewhere. Keep pointers short and move on.
Worked selection sketches
Internal policy Q&A: instruct LLM + RAG; score faithfulness and ACL behavior; do not fine-tune weekly PDFs into weights.
Product image search: multimodal embedding model; measure text→image recall; filter metadata in the index—not hoped-for geometry.
Fraud scoring: specialist predictive model with calibrated probabilities; generative LLM as optional explanation layer only, never as the score.
Edge camera detection: compact detector meeting power/latency budgets; frontier multimodal APIs fail the constraint triangle.
Each sketch picks a function×modality cell first, then a hosting class, then a suite. That order prevents brand-first shopping.
Anti-patterns
One leaderboard to rule all tasks. Mixing embedding models in one index. Evaluating chat models only with vibe checks. Ignoring license IDs. Hot-swapping production models without suite diffs. Using a generative API as a database. Treating open weights as zero-ops. Expanding context windows as a substitute for retrieval engineering.
Operational checklist
Registry entry complete. Private suite exists and has an owner. Scorecard filled with hard constraints. Bake-off recorded. Hosting and residency approved. Canary and rollback defined. Monitoring covers quality, cost, and safety signals appropriate to the family. Deprecation contact list current.
Skip the checklist and you inherit a demo with a billing account.
Closing—model entities should stay linked in the AI entity graph
AI models are selectable artifacts with functions, modalities, eval contracts, and lifecycles. Taxonomy and scorecards make selection accountable. Leave LM mechanics, generative modality maps, and deep architecture pedagogy to their guides; keep this page as the enterprise comparison and lifecycle layer that connects them.
Modality-specific selection notes
Text generative models need suite coverage for instruction following, grounding when retrieval is present, tool-schema validity, and multilingual slices you claim to support. Do not import vision metrics into text bake-offs.
Vision predictive models need camera-domain realism: compression, lighting, resolution, and class priors. Public COCO-style wins transfer poorly to factory floors. Edge constraints may disqualify otherwise strong cloud models—see future edge guidance; here only note that placement is a selection dimension.
Speech models separate recognition and synthesis scorecards. Word error rate is not mean opinion score. Domain accents and telephony codecs matter as much as architecture brand.
Embedding models need metric and dimension contracts matching the index. Mixing models in one collection is a selection failure that shows up as random neighbors—owned operationally with embeddings and vector databases, flagged here as a hard “no.”
Tabular predictive models often belong outside foundation-model fashion. Gradient-boosted trees and calibrated logistics remain legitimate “AI models” in enterprise portfolios. Include them in taxonomy so buyers do not force every problem into a chat API.
Safety and policy models as first-class artifacts
Content filters, prompt-injection detectors, and topic classifiers are models with false-positive economics. Selecting only a generator without policy layers shifts risk to application code. Score safety models on false block rates for legitimate business prompts and miss rates for disallowed classes—jointly with the generator they wrap.
Alignment behaviors inside instruction-tuned LLMs are not a complete safety program. Treat them as one layer. Selection should document which hazards are owned by model refusals versus external filters versus human review—consistent with AI safety hazard thinking without turning this into a safety encyclopedia.
Adapters, ensembles, and routers
Many production systems are not one model: routers send traffic to specialists; ensembles blend scores; adapters specialize a base. Selection then includes routing policy and fallback models. Evaluate the system you will ship, not only the hero checkpoint.
Routers themselves may be small classifiers or heuristic rules. Document them. Silent routing changes create regressions that look like “the big model got worse.”
Fine-tuned adapters inherit base-model licenses and eval debt. When the base upgrades, adapters need revalidation—lifecycle coupling that scorecards must capture.
Procurement questions that belong on the scorecard
Ask for: version pinning and notice periods; training-on-your-data defaults; residency and subprocessors; uptime history; export of logs needed for incidents; known eval contamination statements; and support for private suites under NDA. Price out output-token heavy or image-heavy workloads honestly.
For open weights: license compatibility with your distribution model, required filters, hosting plan, and who owns security patching. Download is not an operating model.
Refuse purchases justified only by press rankings. Demand suite access or the right to run yours.
Team interfaces for model selection
Platform owns registry and serving standards. Applied ML owns suite design per family. Security owns threat review for hosting class. Product owns UX constraints that change cost and latency. Legal owns license sign-off. Ambiguous ownership produces shadow models in personal API keys.
Write a one-page model charter per production surface: allowed data classes, max autonomy, eval owner, hosting class, and kill switch. Charters prevent drive-by swaps.
Failure gallery
Leaderboard shopping without private pairs. Deploying “latest” into regulated flows. Using one multimodal API for detection, OCR, and chat without per-task suites. Quantizing for cost without recall checks. Assuming open weights remove vendor lock-in while your prompts and evals only work on one chat template. Treating generative fluency as calibrated probability.
Each failure maps to a missing scorecard row: task suite, version pin, function clarity, serving config parity, template lock, or calibration requirement.
Comparing models without false precision
Report intervals and slice-level results, not single headline scores. A one-point leaderboard gap inside noise is not a migration warrant. Prefer paired evaluations on the same items with human adjudication where automatic graders disagree.
When two models tie on quality, decide on latency, cost, residency, and operational maturity. Ties are common; pretending otherwise creates churn.
Document known defects alongside scores. A model that fails a rare but critical slice may still win overall averages—averages are not decision policies.
From shortlist to production pin
Shortlist with public signals and smoke suites. Bake off with private suites. Security and license review in parallel. Canary on shadow or small traffic. Pin the winner. Schedule the first re-eval date. Only then raise default traffic.
Skipping canary because “the suite looked fine” is how silent template shifts become incidents. Suites and canaries catch different failures.
Keep the runner-up warm enough for emergency failover when exit drills demand it. A scorecard without a backup model is a single point of failure with paperwork.
Where AI models sit in the Knowledge graph
This page is the selection hub above specialized model and modality guides. It should receive contextual links when published articles discuss “which model” without needing LM or CV internals. As Batch 3 neighbors publish, refresh links to search, speech, and edge only when those pages are live.
Until then, mention hosting deep dives and benchmark design as boundaries—not link targets—so readers never hit draft URLs. Keep the registry mindset: every production surface names a model ID, a suite, and an owner.
Ship taxonomy clarity first; depth lives in the specialist guides. That separation is what keeps a 111-guide library coherent instead of eleven copies of the same essay.
Pin the artifact. Measure the decision. Route the rest. That operating posture outlasts any single model brand and is the point of a dedicated AI models guide inside Brel’s Knowledge library.
References and further reading
- Mitchell, M., et al. (2019). Model Cards for Model Reporting.
- Bommasani, R., et al. (2021). On the Opportunities and Risks of Foundation Models.
- Liang, P., et al. (2022). Holistic Evaluation of Language Models.